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Preface 



This volume is dedicated to Professor Arto Salomaa, a towering figure of 
theoretical computer science, on the occasion of his 65th birthday. His sci- 
entific contributions and influence are enormous - we refer the reader to the 
following Laudation for an account of his scientific achievements. 

The book is a tribute to Arto by the theoretical computer science com- 
munity. It includes the contributions of many outstanding scientists, and it 
covers most of Arto’s many research areas. Due to its representative selection 
of classic topics and cutting edge trends in theoretical computer science it 
constitutes a comprehensive state-of-the-art survey. 

We feel very honored that we have been given the opportunity to edit 
this book - we also feel privileged to be his friends. We are indebted to 
all contributors for paying tribute to Arto through this book. We thank Dr. 
H. Wossner and Mrs. I. Mayer from Springer- Verlag for a perfect cooperation 
in producing this volume. Our special thanks go to Mr. M. Hirvensalo and 
Mr. A. Lepisto, the “2nd generation Ph.D. students’* of Arto, for their work 
on this book. 



Turku, April 1999 Juhani Karhumdki 

Hermann Maurer 
Gheorghe Pdun 
Grzegorz Rozenherg 
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Laudation for Arto Salomaa 



Arto Kustaa Salomaa was born in Turku, Finland, on June 6, 1934. He pub- 
lished his first paper in 1959, and since then he has published about 300 
papers and 11 books, and has edited 15 books - see the following Bibliogra- 
phy. He has had enormous influence on theoretical computer science and is 
certainly one of the founding fathers of formal language and automata theory. 

His influence on theoretical computer science extends far beyond his writ- 
ings. He has supervised 24 Ph.D. students in Finland, Denmark and Canada, 
and many of them have become top scientists in various areas of computer 
science. 

He is on the editorial board of some fifteen international journals and 
book series and has been a member of program committees for almost all 
important conferences in theoretical computer science in Europe. 

Arto is a very stimulating colleague, and he has cooperated with many 
scientists all over the world - his list of co-authors includes some 53 names. 

His contributions to theoretical computer science have been recognized 
in many ways. He must be one of the most decorated computer scientists - 
he has been awarded six Honorary Doctorates (in four countries) and has 
received many scientific awards. He is a member of Academia Europaea, the 
Academy of Sciences of Finland, and the Hungarian Academy of Sciences. 
He was the President of the European Association for Theoretical Computer 
Science (EATCS) from 1979 until 1986, and had an enormous influence on 
this organisation through many other functions. Some of the honors he has 
received are quite unusual. On one of his trips a Maori group north of Auck- 
land put on a Powhiri, a ceremony usually conducted as a way of honoring 
persons of the importance of the Queen of England. Arto may well be the 
only computer scientist to whom such a tribute has ever been paid. 

Prom all the facts listed above it must be clear that in celebrating Arto’s 
65th birthday we pay tribute to a towering figure of theoretical computer 
science. There is no doubt that theoretical computer science owes a lot to 
Arto Salomaa. 

Throughout his scientific career Arto has had many offers from top uni- 
versities all over the world. However, he has decided that Turku is his home 
town and that this is where he wants to have his base. He received his Master 
of Science from the University of Turku in 1956, and although he did his grad- 




XII Laudation for Arto Salomaa 



uate studies in Berkeley, he defended his Ph.D. at the University of Turku 
in 1960. He became a full professor at Turku University in 1966. Since then 
he has made Turku a real place of excellence in theoretical computer science, 
and certainly the world capital in formal language and automata theory. 

Although Turku was always his home bcise, he has traveled extensively. He 
spent long periods of time at the universities of Western Ontario and Waterloo 
in Canada and the University of Aarhus in Denmark. He has also had quite 
lengthy research stays, among others, at the universities of Auckland, Graz, 
Karlsruhe, Leiden, Szeged, and Vienna. 

He has lectured at more than 150 universities and research centers all 
over the world - he is a very inspiring lecturer. His lectures and his books 
demonstrate a unique ability: he is always able to pick out from the sea of 
results the real jewels, and then present them in the way that jewels deserve 
(quite often the jewels are also created by him). As a matter of fact, one of his 
most frequently cited books is “Jewels of Formal Language Theory”. That’s 
why we thought that the title “Jewels are Forever” fits this book very well. 



Turku, April 1999 



Juhani Karhumdki 
Hermann Maurer 
Gheorghe Pdun 
Grzegorz Rozenherg 
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Summary. We study defects affecting state transitions in sequential circuits. The 
fault-free circuit is modeled by a semiautomaton M, and ‘simple’ defects, called sin- 
gle faults^ by a set 5 = of ‘faulty’ semiautomata. To define multiple 

faults from S, we need a binary composition operation, say ©, on semiautomata, 
which is idempotent, commutative, and associative. Thus, one has the free semilat- 
tice 5® generated by 5. In general, however, the single faults are not independent; 
a finite set E of equations of the form © . . . © © ... © describes 

the relations among them. The pair (5, E) is a finite presentation of the quotient 
semilattice /rj, where r] is the smallest semilattice congruence containing E. In 
this paper, we first characterize such abstract quotient semilattices. We then survey 
the known results about random-access memories (RAMs) for the Thatte- Abraham 
fault model consisting of stuck-at, transition, and coupling faults. We present these 
results in a simplified semiautomaton model and give new characterizations of two 
fault semilattices. 



1 Single and Multiple Faults 

In this paper we study the following general problem. We have a fault- free 
object^ which is completely abstract at this stage. When the object is man- 
ufactured, zero or more ‘simple defects’ may be introduced. Each of these 
defects produces a faulty version of the object, called a single fault. Let 
D = {di, . . . , dk} be the set of simple defects and let S = {si, . . . , Sk}, k >0, 
be the corresponding set of single faults. 

If several simple defects are present in the faulty object at the same time, 
we have a multiple fault. Our objective is to characterize such multiple faults 
in terms of the single faults. 

In the simplest case, the single faults are not related in any way. Here, a 
multiple fault is simply a set of single faults. For example, suppose there are 
two single faults. The fault-free object corresponds to the empty set of single 
faults; the objects containing one defect, to {si} and {52}; and the double 
fault, to the set {51,52}. The composition operation on single faults that 
yields multiple faults is set union. Our multiple fault model is the algebra 
( 2 '^,U, 0 ). Note that this algebra is a finite semilattice, with identity 0 and 
zero 5 , generated by the finite set S under the union operation. 
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In practice a set A of defects may result in a faulty object that is indistin- 
guishable from the faulty object corresponding to defects from a different set 
B. We will write A = B to indicate that A and B result in the same object. 
Thus, in general, we have a finite set S of single faults, the set 2^ of potential 
multiple faults, and a finite set £ = {(A = 5) \ A,B £ 2^} of equations. Our 
model for multiple faults is the quotient semilattice 2^ /rjs, where is the 
smallest semilattice congruence containing £. 

2 Finitely Presented Semilattices 

Abstractly we have the following setting for our problem. We are given a 
finite set 5 of generators and a finite set E of equations of the form a = b 
where a and b are words over the alphabet 5. Let 5* be the free monoid with 
identity 1 generated by 5, and let r] be the smallest semilattice congruence on 
S* containing E. Then the quotient semilattice 5* = S* /r] is the semilattice 
defined by the presentation (S^E). For any word u over 5, let [u]rf be the 
//-class of u. The multiplication • on 5* is given by [u]rf •[v]rj = [uv]r). 

For a word u over 5, let a{u) be the set of elements of S appearing in u. 
If u and V are words over 5 such that a{u) = a{v), then u and v represent 
the same element of the free semilattice generated by 5, hence also of S*. 
Thus, via a, the free semilattice with identity generated by S is isomorphic 
with the semilattice (2’^,U,0). For an equation a = b in E^ let a(a = b) 
be the equation a(a) = a(6); moreover, let a(E) be the set of equations 
a(a = b) where a = b is in E. Let 9 be the smallest congruence relation on 
2^ containing a(E). Then the semilattice 5* is isomorphic with the quotient 
semilattice 2^/6. Thus to compute the product of two elements in 5*, it is 
equivalent to compute the product of the corresponding sets in 2^/6. 

In general, there may be several different subsets of S representing the 
same element of 2^/9. However, for every T C S one can choose a canonical 
representative as follows. 

Proposition 1. Let T C S. Then there is a unique set rep{T) in the 9-class 
[T]e ofT such that V C rep{T) for all T £ [T]e. 

Proof. The class [T]^ is closed under union. Hence, the union of all sets 
T' £ \T]e is the desired canonical representative. 

We now give an algorithm for constructing the canonical representative 
rep{T) of any set T C S. Given that a is a semilattice isomorphism, instead 
of words and equations on words, we consider only subsets of 5 and equations 
on sets. An equation = A* = Bi is applicable to T if A* C T or Bi C T. The 
set of equations will be represented by an array f = (ei, . . . , e^n)- Variable £' 
represents equations that have not yet been eliminated. Variable T' represents 
the set obtained from T by the equations used so far. The size of the array £' 
is denoted by '|. An application of an applicable equation to V consists of 
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replacing T* by T*\jAiUBi and of deleting e* from f With these assumptions 
we have the algorithm shown in Fig. 1. 

function rep(5:array[l..m] of equations, T: subset of S): subset of S; 
var S': array of equations, T': subset of 5; 

S' ^S\T' ^ T; 
while S' do 
i ^ 1; 

while a not applicable do 
% i — i -h I5 
if z > \S'\ then 
rep T'; 
exit {rep}] 

S' S' with 6i deleted; T' T' U Ai U Bi] 
rep T' ] 

Fig. 1. Function rep. Scoping is indicated by indentation. 



Let 5'^ be the set of distinct representatives obtained by rep. Let V be 
defined as follows: For any sets A,BeS^,A\/B = rep {A U B). 

Theorem 1. The algebra (5'^, V,0) is a semilattice with identity. The three 
semilattices 5*, 2^/6 and 5^ are isomorphic. 

Proof. The mapping rep from 2^ to induces an isomorphism from 2^/9 
onto 2^^ by /x([T]^) = rep{T) for TCS. 

Note that Theorem 1 provides an algorithm for solving the word problem 
in finitely presented semilattices. The solvability of the word problem itself 
follows already from some very general theorems about finitely presented 
algebras [4]. Surprisingly, however, we failed to find our simple solution to 
this problem in the literature. 

As in any finite semilattice, one can derive a second semilattice operation 
A on such that (*S^, V, A, 0, 5) is a lattice, where 

TAT'= \f U. 

U€S^ ,UCTnT' 

It turns out that T AT' = TflT', that is, the set of canonical representatives 
is closed under intersection. 

Example 1. Let 5 = and 

E = [sitj = Si I i,j € {0, 1}} U {so5i = sositoti}. 

One verifies that 5* = S*/r] has the 7 elements 0, to, t\, toh, sototi, sitoh, 
and soSitoh. The Hasse diagram of 5* is shown in Fig. 2. 
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Fig. 2. Semilattice of Example 1 



3 Faults in Sequential Circuits 

We now make our problem more concrete by considering sequential circuits. 
Such circuits can be modeled by Mealy automata defined as follows. A finite 
deterministic Mealy automaton is a quintuple M = (Q,X, F, J, A), where Q 
is a finite nonempty set called the state set^ X is a finite nonempty set called 
the input alphabet, 5 is a function, called the transition function, from Q xX 
into <5 j set F is a finite nonempty set called the output alphabet, and A is 
a function, called the output function, fi'om Q x X into F. When a defect 
is present in a sequential circuit, we assume that the faulty circuit is still 
represented by an automaton. A faulty automaton may differ from the fault- 
free one in the state set, transition function, and output function. 

In this paper we consider only defects affecting the state set and the 
transition function; hence, we can use semiautomata. A finite deterministic 
semiautomaton is denoted by M = (Q,X,S), where Q, X, and 5 are as above. 
The empty semiautomaton is 0 = (0,X, 0). 

Example 2. We give an example involving partial semiautomata, where the 
transition function is defined only on a subset of Q x X. Consider an ar- 
bitrary partial semiautomaton M = (Q,X,S) and the set S of faulty par- 
tial semiautomata of the following kind. For each transition S(q,x) of M 
there is a faulty semiautomaton Mq^x = such that Sq^x coincides 

with S, except that 6q^x{Q,x) is undefined. The composition of two partial 
semiautomata = (Q,X,S^) and M‘^ = (Q,X,S‘^) with S^,S‘^ C S, is 
© m2 = (Q,X,S^^‘^), where = 6^ n S‘\ Clearly, M is 
the identity element, and the semiautomaton (Q,X,^) with no transitions is 
the zero element. The algebra (5®, 0, M) is isomorphic with the intersection 
semilattice of the subsets of S. Note that this is the free semilattice with 
identity generated by S. 

There has been extensive research on products of automata with out- 
put [5]. However, very little is known about composition operations on semi- 
automata in general. Hence, in the rest of the paper we turn to semiautomata 
with more structure, where several results about multiple faults are known. 
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4 Single Faults in RAMs 

A component semiautomaton is a finite deterministic semiautomaton M = 
(Q,X,S), where Q has the following property: There exists an integer n > 0 
and n finite nonempty sets Qi, . . . , such that Q C Qi x ... x Q^. Thus 
each state g € Q is an n-tuple g = (gi , . . • , gn) » where qi e Qi for i = 1^ ... ,n. 
In the sequel, Q» is assumed to be minimal for each i, that is, for every a G Qi, 
there is q G Q such that g,- = a. 

A component semiautomaton is binary if Qi , . . . , Qn ctre subsets of {0, 1}. 
In this paper we consider only binary semiautomata. 

We use semiautomata to model random access memories (RAMs). We 
assume that a RAM consists of n one-bit cells. The fault- free n-cell RAM is a 
semiautomaton M = (Q,X,S), where Q = {0, 1}”, X = Xi with Xi = 
and S is defined by J((gi, . . . , g^, . . . ,gn),<) = (gi, . . . ,a, . . . ,gn), 
for a G {0, 1}. Here g = (gi, • • . , gn) is the present state of the RAM, and 
S{q,x) is the next state of the RAM after input x G X has been applied. 
Inputs Wq and w\ are the ‘write 0 to cell V and the ‘write 1 to cell V operations, 
respectively^. 

In the sequel, the number n of cells in a RAM is a fixed integer. Also, since 
all the semiautomata considered here are over the alphabet X given above, 
this alphabet will not be explicitly mentioned, and a semiautomaton will be 
denoted simply by M = ((3,(5). One-cell and two-cell RAMs are shown in 
Figs. 3 and 4, respectively; they are labelled M. 

When a RAM is manufactured, a defect resulting in a faulty RAM may 
occur. Many defects that occur in practice can still be represented by normal 
semiautomata [3]. If that is not the case, however, we represent the defective 
RAM by the special empty semiautomaton 0 . 

To model defective RAMs, we associate with the fault-free n-cell RAM 
M = ((3,(5), a family A of semiautomata M' = {Q' ,8'), called delta faults 
or simply faults of M, which may differ from M in the state set and the 
transition function. The state set (3' is a subset of Q and the transition 
function S' is arbitrary. Both M and 0 satisfy the definition of a fault. 

We now give some examples of common faults: the transition, stuck-at, 
and coupling faults. These faults constitute the set T of Thatte- Abraham 
faults [7], and are the main object of our investigation here. 

The complement of a E {0, 1} is denoted by a. A RAM with a transition 
fault in cell Hs a fault M*”’" = ((3, (5*“'‘^), where a G {0, 1} and (5*^" behaves 
like (5 except that, when the input is applied to any state in which cell i 
is in state a, the write operation is unsuccessful, that is, 

^ HQi j • ■ • j Qi—1 j j • • • 5 gn)j ^a) ~ (^1 ? • • ■ 5 Qi—1 5 5 • • • ? Qn)- 

Figure 3 shows the transition fault All operations to other cells are 

correct and not shown. An n-cell RAM has 2n transition faults. 

^ A RAM also has read operations, an output alphabet, and an output function, 
but these concepts are irrelevant to this paper. 
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Fig. 3. Transition and stuck-at faults of a one-cell RAM M. 



For a £ {0, 1}, a stuck-at-a fault in cell Hs a fault 
where = {^ I 9 = (^i , • • • , <li-i , a, ^i+i ? • • • ? 9n)}, and 8^^^ is the restric- 

tion of 8 to X X, except for the transitions under input w^. This input 
fails to write a in cell i, that is, 

. . .,qi-i,a,qi+i,. . ■,qn),‘wy) = (Qu ■ ■ ■ ,qi-i,a,qi+i ,. . . ,g„). 

Figure 3 illustrates the stuck-at fault All operations to other cells are 

correct and not shown. An n-cell RAM has 2n stuck-at faults. 




Fig. 4. Coupling fault in a two-cell RAM M. The first state component 

refers to cell i, the second one to cell j. 



A coupling fault involves two cells, the coupling celli and the coupled cell 
j. When the present states of cells i and j are a and 6, respectively, where 
a, 6 € {0, 1}, and a is written into cell i, then cell j, as well as cell i, changes 
state. The fault is denoted Figure 4 illustrates the fault 

only the states of cells i and j are shown. All operations to other cells are 
correct and not shown. An n-cell RAM has 4n(n - 1) coupling faults. 
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5 Multiple Faults in RAMs 

A ‘simple’ defect represented by a semiautomaton is considered to be a single 
fault. For example, each transition, stuck-at, and coupling fault is considered 
to be a single fault in the literature on circuit testing. Suppose now that 
two defects and D‘^ are present in a RAM, and are represented by semi- 
automata and It should be possible to represent the simultaneous 
presence of both defects by another semiautomaton. What is needed is an 
composition operation on semiautomata, say ©, such that would 

correctly represent the double fault. Although there are cases where it is quite 
obvious how to construct the semiautomaton for the double fault composed 
of and M^, in general this problem is very difficult. 

We make a number of assumptions about the nature of faults. First, when- 
ever the presence of two defects, represented by semiautomata and 
leads to a circuit outside the normal semiautomaton model, we represent the 
resulting circuit by 0; thus = 0. Second, we assume that the com- 

position of any fault M' with a fault outside the semiautomaton model is 
outside the model, that is, 0 © M' = M' ©0 = 0. Third, if defects and 

are represented by the same semiautomaton M', the composite semiau- 
tomaton should be exactly the same as each of the two components. Thus, 
the composition should be idempotent. Fourth, the order in which we con- 
sider two defects when they are both present in the same circuit should be 
immaterial. Thus, the composition should be commutative. Fifth, the order 
in which we compose three defects should not matter; the final result should 
be a circuit with all three defects. Thus, the composition should be associa- 
tive. Fault composition was first considered in [1]. Major results related to 
this paper were presented in [2,3]. For further justification of the assumptions 
above see [3]. 

Altogether, for any given set of single faults, we have the generic hypoth- 
esis that the composition operation should be a semilattice operation, and 
that the semilattice of multiple faults may contain a zero element 0, and, by 
convention, contains the identity element M. 

We now attack the composition problem for the Thatte- Abraham faults. 
A set Q' C (5i X . . . X Qn is called a subproduct if Q' = Q[ x • • • x for 
some sets Q[ C Qi, . . . C Qn. A semiautomaton is called a subproduct 
semiautomaton if its state set is a subproduct. Note that all the faults in T, 
including M and 0 are (binary) subproduct semiautomata. Let V be the set 
of all subproduct faults; from now on, we consider only faults from V. An 
example of a fault that is not in V is the stuck-equal fault of Fig. 5. 

5.1 The Semilattice 

In addition to the generic properties of fault composition, there are also 
conditions derived from the physical nature of defects. We now state a set of 
such physical conditions for the Thatte- Abraham fault model. Condition pi 
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Fig. 5. Stuck-equal fault . 



states that a cell that is both stuck-at-0 and stuck-at-1 results in a fault that 
is outside the semiautomaton model. Condition p 2 affirms that a stuck-at 
fault in a cell is stronger than any transition fault in that cell. Condition 
P 3 declares that a stuck-at fault in a cell is stronger than any coupling fault 
in which that cell is the coupled cell. For a justification of these conditions 
see [3,6]. 

(pi) = 0, 

(P2) 0 

Ips) 0 = M*-". 



Our main problem now is to find a binary composition operation on 
semiautomata that satisfies both the generic conditions and the physical 
conditions above. Let M = ((3,(5) be the fault-free n-cell RAM, and let 
and be two faults of M. We define the compo- 

sition oM^ to be the semiautomaton = ((3^°‘‘^,(5^°^), where (3^°^ = 
D (3^, and is defined as follows. Consider a state q = (gi, . . . , gn) ^ 
and an input x G Xi. Let S{q,x) = (pi, . . . ,pn), = (pj,. . . ,pi), 

S‘^{q,x) = (Pi,...,Pn), and (5^°^(g,x) = (Pl°^ ■ . . ,Pn°^). To determine 
one applies the rule: 

Circle Rule. For all j, (a) = a, if (3]°^ = {a}, and (b) p]°^ = pj © 

[{pj 0p]) -f {pj 0pj)j, if = {0, 1}, where 0 and 0 denote the inclusive 
and exclusive OR operations, respectively. 

In the following theorem, (T°,o) denotes the semilattice generated by T 
under the operation o. 

Theorem 2. The algebra (P, o) is a semilattice with 0 as zero element and 
M as the identity, and o satisfies pi-p3. The algebra (T°,o) is a subsemi- 
lattice of (V, o), and pi-ps form a minimal set of defining relations for T°. 
Furthermore, T° is a proper subsemilattice ofV. 

Proof The first two claims of the theorem were proved in [2]. In that paper, 
delta faults were defined more narrowly, since they were required to have 
state sets which were subproducts of Q. Thus the family of delta faults in [2] 
is precisely V. Semiautomaton of Fig. 6 is not in T°. This follows from 
the characterization of T° given in Theorem 4 in Section 6. 
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Fig. 6. Faults and 



5.2 The Semilattice 

A different circuit implementation of a RAM may lead to a different set of 
physical conditions. In fact, in another useful fault model the composition 
must satisfy, in addition to pi-ps, also p^ and ps, where 



(P4) 0 

(ps) O = M*=“. 

The fourth condition requires that a transition fault be stronger 

than any coupling fault The fifth condition states that the stuck-at 

fault is stronger than any coupling fault We again refer the 

reader to [3,6] for the justification of these conditions. One can verify that ps 
is a consequence of pi~p 4 . 

We now define a second composition operation o as follows. Let M = 
(Q, (5) be the fault-free n-cell RAM, and let = (Q^ and 
be two faults of M. We define the composition o to be the semiau- 
tomaton where r\Q^ and is defined cts 

follows. Consider a state g = (^i, . . . ,^n) ^ and an input x e Xi. Use 
the same notation as in the Circle Rule, and let 
To determine one applies the rule: 

Diamond Rule. If p] = p? = Pi, then (a) for all j including i, pj^‘^ = a, 

if ^ plo2 ^ p, ^ [(p^. 0pl) + (p. ©p2)], if Q1 o2 = |0, 1}. 

Otherwise, S^^‘^{q,x) = q. 

The algebra (V, o) is an idempotent and commutative grupoid with 0 as 
zero element and M as the identity, but it is not associative. The verification 
of idempotency and commutativity is routine. Figure 6 shows two faults 
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and Letting = M", and = M**, we find that the transition 

from state 10 under input w{ results in = 1 and = 0. 

Since our main interest is in the Thatte- Abraham faults, we add a restric- 
tion to obtain a class of faults in which o is associative. 

A RAM delta fault M' = is change-attempt-activated if for all 

q £ Q', i £ {!,.. . ,n}, and x £ Xi, 

Si{q,x) = Qi=> S'{q,x) = q. 

In such a fault, no change of state is possible, unless the state of the addressed 
cell is being changed in the fault-free RAM. Let A be the set of change- 
attempt-activated faults. 

Let C be the set of delta faults satisfying, for all q £ Q', i £ {1, . . . ,n}, 
and X £ Xi, 

Si(q,x) 7 ^ qi = S'i(q,x) 6'{q,x) = q. 

Let <S = >tnC be the set of change-success-activated faults. One verifies that 
all the faults in T are in S. 

In the theorem below, (T^,o) denotes the semilattice generated by T 
under the operation o. 

Theorem 3. The algebra {S OV,o) is a semilattice with 0 as zero element 
and M as the identity, and o satisfies p\-p^. The algebra (T^,o) is a proper 
subsemilattice of {S C\V,o). Furthermore, equations pi~p 4 form a minimal 
set of defining relations for . 

Proof. The diamond operation was introduced in [3], as an composition op- 
eration on delta faults. However, we later discovered that this operation was 
ill-defined for some delta faults; consequently, several results in [3] are incor- 
rect as stated. The subproduct condition should be added, as it is done in 
the present paper, to obtain correct statements. With this added condition, 
the proofs of [3] apply for the first two claims of the theorem. The proof 
of the third claim is similar to the proof of the corresponding statement in 
Theorem 2. 

6 Characterization of T° and 

Roughly speaking, a RAM delta fault involving a set C of cells is pattern- 
sensitive if it occurs only for some values of the cells that are not in C. 
More precisely, A delta fault M' = {Q' ,5') is pattern-sensitive if one of the 
following conditions holds: 

(1) There exist i £ {1, . . . ,n}, x £ Xi, and q, g' £ Q' such that q^ = qi and 

S{(q,x) 5[{q\x). 

(2) There exist i,j £ {!,..., n}, x £ Xi, and q,q' £ Q' such that q[ = qi, 
q'j = qj, and S'j{q,x) ^ S'j{q^,x). 

Otherwise, it is pattern-insensitive. Let I be the set of pattern-insensitive 
faults. 
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Lemma 1. The sets A, S and X are closed under o and o. 

Lemma 2. Let E A with and M‘^ = and 

let M = (Q,S) be the corresponding fault-free automaton. 

1. For all q G and all x £ X, if S^{q,x) = S{q,x) and 6‘^{q,x) G 
then 

2. If € C then, for all q G and all x £ if 6^{q,x) = S(q,x) and 

S^{q,x) £ then = S‘^{q,x). 

Theorem 4. (a) A delta fault is change- attempt- activated, pattern-insensi- 
tive, and a subproduct fault if and only if it is equivalent to a o-composition 
of Thatte- Abraham faults, that is, T° = AC\X C[V. (b) A delta fault is 
change-success-activated, pattern-insensitive, and a subproduct fault if and 
only if it is equivalent to a o-composition of Thatte- Abraham faults, that is, 

T^ = cr\T^. 



Proof. By the definition of T one has T C ACiX (IV C\C. By Lemma 1 and 
Theorems 2 and 3, AOXOV is closed under o and AnX DV C\C is closed 
under o. Therefore, T° C AC\X C\V and C AOX DV DC. 

We first show that T° "D AC\X CiV. Consider M' = {Q',S') £ AC\X C\V. 
As M' G A, errors can arise only from writing a new value. Let cr{M’) be 
the number of state set components Qj which are singleton sets; let r{M') 
be the number of pairs (i, a) with i G {1, . . . , n} and a £ {0, 1} such that, for 
some q £ Q' with qi = a, one has S[(q,wl) ^ a; let j{M') be the number of 
quadruples (i, a,j, b) with i,j E {1, . . n}, i ^ j, and a,b £ {0, 1} such that, 
for some q £ Q' with qi = a and qj = b, one has Sj{q, wl) = b. 

We show that M' can be represented as M' = S oT oC o M where 5 is a 
o-composition of stuck-at faults, T is a o-composition of transition faults, and 
C is a o-composition of coupling faults. In all three cases, the o-composition 
of zero faults is M. 

We first determine the component S. By splitting 5 off, we may then 
assume that cr(M') = 0. We then determine the component T. By splitting 
T off, we may assume that also r(M') = 0. Third, we split C off, leaving a 
semiautomaton M' with cr{M') = r(M') = 7 (M') = 0. One verifies that this 
implies M' = M, given that M' £ AC\XC\V. We proceed by induction, first 
on cr{M'), then on r(M'), and then on 7 (M'). If M' G T or M' = 0, nothing 
needs to be proved. Assume, therefore, that M' ^ T and M' ^ 0. 

First suppose that cr(M') > 0. Then Qj = {a} for some j and some a. 
Let and let be defined as follows. Let 

=Q[x - x (5'_1 X {0, 1} X g'+i X • • • X g'„. 

For q & Q^, let q' = (qi,.. . ,qj-i,a,qj+i, . . . ,?«)• For all inputs x and for all 
* 7^ (9)®) = For component j, Sj(q,x) is q,, \f x = , and q, 

otherwise. 
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We verify that oM^ = M'. Clearly, Q' = nQ‘^. Consider q £ and 

X £ X . If X ^ Xj then 6^{q,x) = 6{q^x) and S‘^{q,x) £ Q'] hence S^^^{q,x) = 
6‘^{q,x) by Lemma 2 and x) = S'{q,x). Now consider x E Xj. If x = 
then S'{q,x) = q as £ A and (S^°‘‘^(g,x) = q = S'{q,x). Now let x = wL. 
Then, by the definition of o, S^°‘^{q,x) = 6'{q,x). 

By construction, £ AC\X C\V and cr(M^) < a{M^). Hence, by in- 
duction, can be written in the form = M" o M'" where M" is a 
o-composition of stuck-at faults and M'" £ AOlnV with cr{M'”) = 0. Then 
(M^ o M”) o M'" is the required representation of M' with 5 = o M" , 

Now we consider the case of a{M') = 0. Then Q* = Q = {0, 1}’^. Suppose 
that r(M') > 0. Then there is a cell i such fails, that is, there is a state 
q with qi = a such that S[(q^w\) = a. Let and let = (Q,(J^) 

be defined as 

x2/_ 

\{S'i{q,x),...,6'i_j^{q,x),a,S'i^-^{q,x),...,S'„{q,x)), if x = < , 

for gf E Q and x £ X. Using the fact that M' £ A C\X one verifies that M‘ = 
By construction, E AC\XfXP^ a{M‘^) = 0, and r{M‘^) < r(M') as 
M' E X. Hence, by induction, M‘^ can be written in the form 
where M" is a o-composition of transition faults and M'" £ AC\XC\V with 
cr(M'") = 0 = r{M‘”). Then (M^ o M") o M'" is the required representation 
of M' with T = M^o M". 

Finally, assume that a(M') = r(M') = 0 and 7(M') > 0. Then there are 
distinct cells i and j an^ values a, 6 E {0, 1} such that for some state q £ Q 
with qi = a and qj — b one has 6j{q,wD — b. Let and let 

= (Q, (S^) be given by 



Si 




b, 



if X = It;* , qi = a, qj = 6, and k = j, 
otherwise, 



for gf E Q, fc = 1, . . . , n, and x £ X. We prove that M' = o M‘^. 

Clearly, Q' = = Q. Consider q £ Q and x E X. If x 7^ ?;;* or ^ a 

or qj ^ b then S^{q,x) = S{q,x); hence S^°^{q,x) = S^{q,x) = S'{q,x). 

Now suppose that x = qi a and qj = b. Then 







<5* (9, a:), 

b, 



if # j, 
if fc = j, 



for = 1, . . . ,n. As M' E X, this is equal to (5J.(g,x). 

By construction, E 7^, tr(M^) = 0 = r(M'), and 7(M^) < 

7(M'). Hence, by induction, can be written in the form of a o-composition 
of coupling faults and, therefore, also M' can be represented in this way. This 
completes the proof of statement (a). 

The proof of the inclusion T^DAOXOVilC is analogous to the one 
for o except that the faults to be considered have to be also in C at every 
induction step. 
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Proposition 2. There is a homomorphism of onto such that its re- 
striction to is the identity mapping. For if o E C, 

then = 

Proof. The two semilattices have equations pi-ps in common. 
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Summary. A finite-state machine is called a Thompson machine if it can be con- 
structed from a regular expression using Thompson’s construction. We assign dis- 
tinct bracket pairs to the source and target states of all machine units that make 
up a Thompson machine. Each path from the start state to the final state of such a 
Thompson machine spells out a Dyck string. The collection of all such Dyck strings 
is the Thompson language of the given machine and the collection of such strings 
that are spelled out by simple paths is the simple Thompson language of the given 
machine. We characterize simple Thompson languages and Thompson languages, 
and we investigate their relationship. 



1 Introduction 

In 1968, Thompson [6] introduced an elegant and simple inductive construc- 
tion of a finite-state machine from a regular expression. The resulting finite- 
state machine has size linear in the size of the original expression and each 
state has no more than two ingoing transitions and two outgoing transitions. 
A resurge of interest in the implementation of machines has resulted in some 
new discoveries about the Thompson construction [2,4]. 

Recently [5] we characterized the underlying digraphs of the machines re- 
sulting from the Thompson construction, Thompson digraphs and Thomp- 
son machines, respectively. The characterization is diflFerent from the one of 
Caron and Ziadi [3] for Glushkov digraphs although it is necessarily related 
to theirs indirectly. 

The development of the characterization of Thompson digraphs [5] led 
us to the idea of associating labeled left and right brackets with the source 
and target states, respectively, of the unit machines that are used in the 
inductive Thompson construction. (We use the designation source state and 
target state when the corresponding states axe not a start state and not a final 
state which occurs when we discuss a unit that is part of a larger machine.) 
We can view the bracketing as a means of linearizing the induction tree given 
by the Thompson construction. Once we have labeled the states in this way 
each path from the start state to the final state spells out a Dyck string. 
The set of all Dyck strings spelled out by such paths is a subset of the Dyck 
language for the appropriate alphabet and we call it a Thompson language. 




Thompson Languages 17 



Although Thompson languages appear to be nonregular, they are regular 
languages as they are specified by the duals of finite-state machines. We 
establish a Kleene-like theorem for them that is analogous to the standard 
Kleene theorem for regular languages; see Section 4. 

In Section 5, we characterize simple Thompson languages. The set of 
all strings spelled out by simple paths from the start state to the final state 
of a Thompson machine is a simple Thompson language. We then character- 
ize Thompson languages in terms of simple Thompson languages — our main 
result. 

2 Notation and terminology 

We recall the basics of finite-state machines and regular expressions and 
introduce the notation that we use. 

A finite-state machine^ consists of a finite set Q of states, an input 
alphabet i7, a start state s G Q, a final state f G Q and a transition rela- 
tion S C Q X Ex X Q, where A denotes the null string and E\ = Z* U {A}. 
Clearly, we can depict the transition relation of such a machine as an edge- 
labeled digraph (the labels are symbols from E\); it is usually called the state 
or transition digraph of the machine. 

Let be an alphabet. Then, we define a regular expression E over E 
inductively as follows: 

£7 = 0, where 0 is the empty-set symbol; 

£7 = A, where A is the null-string symbol; 

£7 = a, where a is in E\ 

E — (F -I- G), where F and G are regular expressions; 

E = {F • G)^ where F and G are regular expressions; 

£7 = (F*), where F is a regular expression. 

We define the language of E inductively, in the usual way [1,7]. 

We say that a regular expression E is empty free if E does not contain 
any appearance of the empty-set symbol. 

3 Thompson machines and languages 

Thompson designed a novel method [6] to compile regular expressions into 
a form that is suitable for pattern matching in text files. The construction 
is defined inductively (see Fig. 1). We treat only empty-free regular expres- 
sions; therefore, we never have to consider the finite-state machine given by 
Fig. 1(a). The Thompson construction gives a finite-state machine that has 
a number of ple 2 ising properties: 

^ Normally, a finite-state machine is allowed to have more than one final state 
and, sometimes, more than one start state. The formulation we have chosen is 
appropriate for the study of Thompson machines. 
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t) o 



(a) 



(b) 



(c) 




X 




Fig. 1. The Thompson construction. The order of the figures corresponds to the 
order of the cases in the definition of regular expressions. The finite-state machines 
correspond to the regular expressions: a. F = 0; b. £7 = A; c. £7 = a, a € X"; d. 
£; = (F -h G); e. £; = (F . G); and f. F = (F*). 



1. It has an even number of states. 

2. It has one start state that is only exiting; that is, there are no transitions 
that enter it. 

3. It has one final state that is only entering; that is, there are no transi- 
tions that leave it. 

4. Every state is on a simple path from the start state to the final state. 

5. Every state has at most two ingoing transitions and at most two outgoing 
transitions. 

6. Its size is at most three times the size of the given regular expression. 

In Fig. 2, we give the result of the Thompson construction on the regular 
expression (((a + b)*) • ((6 + A) • a)). 

We define a Thompson machine to be a finite-state machine that is 
obtained by the Thompson construction from an empty-free expression. We 
name the units that the Thompson construction uses to assemble a Thomp- 
son machine as follows: the base units (Figs. 1(b) and (c)); the plus unit 
(Fig. 1(d)); the dot unit (Fig. 1(e)); and the star unit (Fig. 1(f)). In Fig. 1, 
the source and target states for each unit correspond to their start and final 
states, respectively. Observe that the uniqueness of the start and final states 
implies that each Thompson machine has an even number of states. 
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X 




Fig. 2 . The result of the Thompson construction on the running example expression 
(((a + 6)*). ((6 + A), a)). 



We now define Thompson strings and languages. Define a countably in- 
finite alphabet F of bracket pairs that we enumerate as . . . , — 

For each n > 0, we define Fn = : 1 < i < n}. Given a language L 

over Fn, the image of L under an injection (or renaming) h : Fn — >F, 
namely h{L), clearly has the same structure as L. (Indeed, this observa- 
tion holds for any injection or bijection of L into an appropriately sized 
alphabet.) Thus, when we write L, we really mean the family L of languages 
{h{L) : ft is an injection Fn — >F}; all languages in this family are struc- 
turally equivalent. 

Given a Thompson machine M that has 2n states, we label the states 
arbitrarily but uniquely with brackets from Fn in the following way: For each 
unit submachine of M, we label its source state with left bracket [i and its 
target state with right bracket ]i, for some i, 1 <i <n. 

Having labeled the states of M with the 2n brackets in Fn, we define a 
Thompson string as any string that is spelled out by some path in M from 
its start state to its final state. For example, if we label the source and target 
states of the machine of Fig. 2 with [i, . . . ,]s as indicated, the strings 

[i [2 [3 ]s ]2 [2 [4 ]4 ]2 ]i [5 [e ]e ]s [s ]s 



and 

[l[2[4]4]2]l[5[6]6]5[8]8 

are Thompson strings of the given machine. Clearly, Thompson strings are 
Dyck strings over however, not all Dyck strings over Fn are Thompson 
strings. Two simple examples are the strings x = [1 ]i [1 ]i and y = [1 [1 ]i ]i; 
we leave the proofs of their non-Thompson-ness to you. Similarly, we define a 
simple Thompson string as any string that is spelled out by some simple 
path in M from its start state to its final state (recall that a simple path 
visits each state at most once). Thus, each simple Thompson string contains 
at most one pair of each of the n bracket pairs. Of the two preceding examples 
of Thompson strings, the first string is not a simple Thompson string, but 
the second string is. 

Given a Thompson machine M, its Thompson language is the set of 
all Thompson strings given by M. Similarly, M’s simple Thompson lan- 
guage is the set of all simple Thompson strings given by M. By definition. 
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every simple Thompson language is finite and some Thompson languages are 
infinite. Clearly, each Thompson language L C is a subset of the cor- 
responding Dyck language D{rn). That each Thompson language does not 
contain all Dyck strings follows from the previous examples of Dyck strings 
that are not Thompson. But, Thompson languages are even more restrictive. 

We investigate the structure of Thompson and simple Thompson lan- 
guages and characterize them in terms of their structural properties. 

4 A Kleene Theorem 

Let T and ST denote the families of Thompson and simple Thompson lan- 
guages, respectively. We introduce variants of the usual regular operations 
that lead to a Kleene theorem for the two families. Given two Thompson 
languages K C D{rm) and L C D{rn), we define bracketed versions of the 
regular operations as follows: 

Bracketed union 0: 

K ^ L = U }, 

where h : Fn — : m -f 1 < z < m + n} is a renaming. 

Bracketed product ©: 

KGL = Kh{L), 

where h : Fn — : m + 1 < z < m -h n} is a renaming. 

Bracketed star ©: 



if® = {u+i}if*{]„+i}. 

Similarly, given two simple Thompson languages K C D{Fm) and L C 
D{Fn), we use the bracketed versions of union and product but we replace 
bracketed star with bracketed prestar defined as follows: 

Bracketed prestar 0: 



The proof of the following theorem is straightforward. 

Theorem 1. The family T of Thompson languages is closed under bracketed 
union, bracketed product and bracketed star. 

The family ST of simple Thompson languages is closed under bracketed 
union, bracketed product and bracketed prestar. 

As a result of the preceding theorem we can provide an inductive defini- 
tion of Thompson languages that parallels the inductive definition of Thomp- 
son machines. Given an alphabet we can inductively define a Thompson 
language as follows: 




Thompson Languages 21 



1. The language {[i ]i} C is a Thompson language. 

2. If K C D{rm) and L C D{rn) are Thompson languages, then K 0 L is 
a Thompson language. 

3. If /C C D{Fjn) and L C D^Fn) are Thompson languages, then JFf 0 L is 
a Thompson language. 

4. If X C D{Fm) is a Thompson language, then is a Thompson lan- 
guage. 

If we replace 0 with 0, then we obtain an inductive definition of simple 
Thompson languages. 

The basis for these definitions is the following Kleene theorem. 

Theorem 2. The family T of Thompson languages is the smallest family of 
languages that contains {[i ]i} and is closed under bracketed unions bracketed 
product and bracketed star. 

The family ST of simple Thompson languages is the smallest family of 
languages that contains {[i ]i} ond is closed under bracketed union j bracketed 
product and bracketed prestar. 

5 The characterization theorem 

We develop a characterization of Thompson languages by first considering 
simple Thompson languages. We present some necessary conditions for a 
language L C D{Fn) to be a simple Thompson language or a Thompson 
language. 

Property 1: Bracket uniqueness. Each pair [j,]j appears at most once in 
each string in a simple Thompson language L. 

The following four properties apply to both simple Thompson and Thomp- 
son languages. 

Property 2: Starting and ending uniqueness. All strings in L begin with 
the same left bracket [i, say, and end with the same right bracket say. 

Given a Thompson machine M of 2n states that axe labeled with the 
brackets in Fn^ let L C D{Fn) be its simple Thompson or its Thompson 
language. Given a Thompson language L C D{Fn), we define L{i) to be the 
set of all strings that appear between the brackets [i,]j in strings of L. (Note 
that the brackets may appear more than once in Thompson strings, but each 
appearance can contain only the same substrings.) More formally, 

L{i) = {x : there are strings w and y such that 

w[ix]iy £ L and x does not contain [i or ]j}. 

Similarly, we define L[i] to be the set of all substrings of strings in L that 
begin with [i and end with ]j; that is, 

L[i] = {[ix]i : there are strings w and y such that 
n)[ix]iy £ L and x does not contain [j or ]j}. 
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Property 3: Unit conditions. 

1. If a pair [i,]i labels a base unit, then L{i) — {A}. 

2. If a pair [t,]j labels a star unit, then L{i) D {A}. 

3. If a pair [i,]^ labels a plus unit, then L{i) ^ 0 and {A} ^ L{i). 

We consider the restrictions on substrings that occur with plus and star 

units: 

Property 4: Multiplicity conditions. 

1. If a pair [f,]i labels a star unit, then all strings in L{i) — {A} begin 
with some bracket [/, j and end with some bracket k ^ i. 

2. If a pair [i,]i labels a plus unit, then all strings in L(i) begin either 
with \j or with [m, where i ^ i ^ m and j ^ m, and end either 
with ]k or with ]n, respectively, where i ^ k, i ^ n and k ^ n. 

The next property captures the sequential nature of the product of two 
machines: 

Property 5: Product sequentiality. Let the pairs [i,]t and \j,]j label two 
units in M that have been combined by product, then, for all strings 
u, Vj w and x, neither u[iv]i[nw]nX nor u[mv]m[j'^]jX are in L, where i ^ m 
and j ^ n. 

The following result follows immediately from the definitions of the prop- 
erties. 

Lemma 1. Let L he a simple Thompson language. Then, L satisfies Prop- 
erties 1, 2, 3, 4 o.nd 5. 

Let L he a Thompson language. Then, L satisfies Properties 2, 3, 4 o,nd 
5. 



We are now in a position to characterize simple Thompson languages. 

Theorem 3. A language L C D{Pn) is a simple Thompson language if and 
only if L satisfies Properties 1, 2, 3, 4 5. 

Proof. We already observed, by Lemma 1 that, if L is a simple Thompson 
language, then it satisfies Properties 1, 2, 3, 4 and 5. We now prove the 
converse. Let L C D(Pn) be a language that satisfies Properties 1, 2, 3, 
4 and 5. To demonstrate that L is a Thompson language, we sketch the 
construction of a Thompson machine M such that the Thompson language 
of M is isomorphic to L. The proof is by induction on the size n of the 
alphabet Pn of L. 

Basis: In this case n = 1, and L = {[i ]i} by Properties 1 and 2. 

Induction hypothesis: We assume the result holds for all languages L C 
D{rm) that satisfy Properties 1, 2, 3, 4 and 5, where 1 < m < n, for some 
n > 1. 
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Induction step: Consider a language L C D(r'n+i) that satisfies Proper- 
ties 1, 2, 3, 4 and 5. Since n > 1, the strings in L must contain at least two 
different brackets. Consider a string x in L. Then, either x begins with a left 
bracket [i and ends with the corresponding right bracket ]i or it does not. In 
the first case, all strings in L must begin and end the same way by Property 2; 
therefore, by Property 3, the strings correspond either to a bracketed union 
or to a bracketed prestar (they cannot correspond to a base unit because 
that would imply that n + 1 = 1, which contradicts the assumption that 
n -h 1 > 2). By Property 3, we can uniquely identify which alternative the 
strings correspond to; therefore, we can replace L either with two languages 
each with fewer brackets than L or with one language that has fewer brackets 
than L. A corresponding Thompson machine (with state labels but without 
edge labels) can be constructed inductively in this case. 

In the second case, x starts with a left bracket [i but ]i does not end x; 
therefore, we can partition x into uv^ where u begins with [i and ends with ]j, 
by the structure of these strings. Since v ^ A, we have discovered a product 
decomposition; namely, by Property 5, there is a left bracket [i such that, for 
all strings x in L, x can be decomposed into strings u and v such that x = uv^ 
u begins with left bracket [i and ends with ]i, and v ^ X. This decomposition 
of strings induces a decomposition of L into two languages J and K such 
that J = L[i] and K = {v : uv £ L and u £ L[i]}. Both J and K have fewer 
brackets than L and both satisfy Properties 1, 2, 3, 4 and 5; therefore, we 
can inductively construct a Thompson machine that corresponds to L. 

The proof of the preceding theorem provides an inductive construction of 
a Thompson machine M (with state labels but without edge labels) from a 
simple Thompson language L such that the simple Thompson language of M 
is L. Based on this construction, we now characterize Thompson languages. 

First, observe that, for each Thompson language L, the simple Thompson 
subset 5 of L consists of all the strings in L that satisfy Property 1. Sec- 
ond, observe that if the bracket pair [i, ]j corresponds to a bracketed prestar, 
then, conceptually, we can replace every string from S[i] in strings of S with 
[t5(i)*]i. We can then repeat this expansion for each bracketed prestar in 
turn. Unfortunately, after the first expansion, we obtain an infinite language. 
Therefore, we construct the Thompson machine Ms for the language 5, where 
each bracketed prestar gives a star unit. The Thompson language of Ms is 
the language L if and only if L is a Thompson language. This proof sketch 
provides our last result, the sought characterization of Thompson languages. 

Theorem 4. Given a language L C D{rn) and its simple Thompson sub- 
set S, L is a Thompson language if and only if L satisfies Properties 2, 3, 4 
and 5, and the Thompson language of Ms is L. 

Observe that the characterization is effective if we have an effective spec- 
ification of L. 
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On Some Special Classes of Regular 
Languages* 



Balazs Imreh and Masami Ito 



Summary. In [5], the directable non deterministic automata are studied. For non- 
deterministic automata, the directability can be defined in several ways. In each 
case considered, the directing words constitute a regular language and one can study 
the families of such regular languages. In this paper, six classes of regular languages 
are defined in accordance with the different definitions of directability given in [5], 
and we investigate the properties of the classes considered. 



1 Introduction 

An input word of an automaton is called directing or synchronizing word if it 
brings the automaton from every state into the same state. An automaton is 
directable if it has a directing word. The directable automata and directing 
words have been studied from different points of view (see [2], [3], [4], [7], [9], 
[10], for example). For nondeterministic (n.d.) automata, the directability 
can be defined in several ways. We study here the following three notions of 
directability which are introduced in [5]. Namely, an input word w of an n.d. 
automaton A is 

(1) D 1-directing if the set of states aw in which A may be after reading 
w consists of the same single state c whatever the initial state a is; 

(2) D2-directing if the set aw is independent of the initial state a; 

(3) D3-directing if there exists a state c which appears in all sets aw. 

It is worth noting that the Dl-directability of complete n.d. automata 
was already studied by Burkhard [1], where he gave an exact exponential 
bound for the length of minimum-length D 1-directing words of complete n.d. 
automata. Here, we study the classes of languages consisting of D1-, D2- 
and D-3 directing words of n.d. automata and complete n.d. automata. The 
next section contains a general preliminaries and the formal definitions of 
these languages, and Section 3 presents some properties of the languages 
considered. 

* This work has been supported by the Hungarian National Foundation for Scien- 
tific Research, Grant T 014888, the Ministry of Culture and Education of Hun- 
gary, Grant FKFP 0704/1997, the Japanese Ministry of Education, Mombusho 
International Scientific Research Program, Joint Research 10044098 
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2 Preliminaries 

Let X be a finite nonempty alphabet. The set of all (finite) words over X is 
denoted by X* and the empty word by e. The length of a word w is denoted 
by \w\. 

By an automaton we mean a system A = (A,X,S), where A is a finite 
nonempty set of states, X is the input alphabet, and S : A x X ^ A is the 
transition function. The transition function can be extended to A x X* in 
the usual way. By a recognizer we mean a system A = (A, X, S, ao,F), where 
(A,X,S) is an automaton, ao(E A) is the initial state, and F{C A) is the set 
of final states. The language recognized by A is the set 

L{A) = {w £ X^ : 8{ao,w) £ F}. 

A language is called recognizable, or regular, if it is recognized by some rec- 
ognizer. The set of all recognizable languages over the alphabet X is denoted 
by Rec(X). 

An automaton A = (A,X,6) can also be defined as an algebra A = 
(A,X) in which each input letter x is realized as the unary operation : 
A — )> A, a ^ S{a,x). Now, nondeterministic automata can be introduced as 
generalized automata in which the unary operations are replaced by binary 
relations. Thus, by a nondeterministic (n.d.) automaton we mean a system 
A = (A, X) where A is a finite nonempty set of states, X is the input alphabet, 
and each letter x{£ X) is realized as a binary relation x'^(C A x A) on A. For 
any a £ A and x £ X, ax'^ = {b £ A \ {a,b) £ x-^} is the set of states into 
which A may enter from state a by reading the input letter x. For any C C A 
and X £ X, we set Cx-^ = {Jlax"^ : a £ C}. This transition can be extended 
for arbitrary w £ X* and C C A. If w £ X^" and a £ A, let = {a}^^. 
This means that ifw= xiX 2 • - - Xk, then = x'^x '2 • • • x'^{C A x A). When 
A is known from the context, we usually write simply aw and Cw for aw"^ 
and Cw-^, respectively. 

An n.d. automaton A = (A,X) is complete if ax'^ ^ 0, for all a € A 
and X £ X. Complete n.d. automata are called c.n.d. automata for short. In 
what follows, we denote a deterministic automaton by A = (A,X, 5) and a 
nondeterministic automaton by A = (A,X). 

The idea of directability of deterministic automata can be extended to 
n.d. automata in several nonequivalent ways. In [5], three definitions are 
introduced as follows. Let A = (A,X) be an n.d. automaton. For any word 
w £ X* we consider the following three conditions: 

(Dl) (3c£A){\/a£A){aw = {c}y, 

(D2) (Va, b £ A) {aw = bw)\ 

(D3) (3c £ A)(Va £ A) {c £ aw). 

If w satisfies (Di), then ly is a Di-directing word of A {i = 1,2,3). For each 
i = 1,2,3, the set of Di-directing words of A is denoted by Dj(A), and 
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A is called Di-directahle if T>i(A) 0. It is proved (see [5]) that Dj(^) is 
recognizable, for every n.d. automaton A and i, i = 1,2,3. The classes of 
Di-directable n.d. automata and c.n.d. automata are denoted by Dir(i) and 
CDir(i), respectively. 

Now, we can define the following classes of languages: For i = 1, 2, 3, let 



^ND(i) = : A e Dir(i)}, 



>CcND(i) = {^i{A) : A G CDir(z)}. 

Since all of the languages occuring in the definitions above are recogniz- 
able, the defined classes are subclasses of Rec(X). 

Finally, let us denote by D the class of directable deterministic automata, 
and for any ^ C D, let D(>I) be the set of the directing words of A. Further- 
more, let 



Cb = {B(A) :AgB}. 



3 Some properties of the languages of directing words 
of n.d. automata 

In [5], the following observation is presented, which is used below. 

Lemma 1 . For any n.d. automaton A = (A^X), D 2 {A)X* = \) 2 {A). If A is 
complete, then X*Di(>l) = Di(^), X*D 2 (. 4 )X* = Il 2 {A), and X^T)^{A)X^ 
= D3(^). 



The following assertion gives a characterization of £d ■ 

Proposition 1 . For a language L C X* , L £ Cb if and only if L L is 
regular, and X*LX* = L. 

Proof. To prove the necessity, let us suppose that L G £d- It is known, (c/. 
[4]) that then L / 0 and L is regular. On the other hand, it is easy to see 
that if li; is a directing word of an automaton A^ then pwq is also a directing 
word of A for every p,q E X*. These observations yield the necessity. 

In order to prove the sufficiency, let us suppose that L ^ i/}, L is reg- 
ular, and X*LX* = L. Since L is regular, there exists a recognizer A = 
(A,X,S,ao, F) such that L = L{A). Without loss of generality, we may as- 
sume that the automaton A — {A,X,S) is generated by one element, namely 
by uq. Since L 7 ^ 0, we have that F 7 ^ 0. On the other hand, by our as- 
sumption, X*LX* = L, which implies that S(a,x) £ F, for every a£ F and 
X £ X, i.e., (F,X,S) is a subautomaton of A where S denotes the restriction 
of S to F X X. Let us denote by A‘ the Rees factor automaton defined by 
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A — {AjF, X, S*) where A/F = (A \ F) U {F} and for every a G A\F and 
xGX, 



and 




if S{a^x) G A\F, 
if S{a,x) G F. 



S\F,x) = F. 

Then, it is easy to see that L = L{A') where A' = (A/F,X, J', oq, {F}). 

Now, we show that A' = (A/F,X, J') is a directable automaton and 
D(A') = L. Indeed, let w G L and a G A/F be arbitrary elements. Since A 
is generated by ao, A' is also generated by ao- This implies that there is a 
word p G X* such that (5'(ao,p) = a. Since = L, pw G L, but then, 

S'{ao,pw) = F, and thus J'(a, w) = F. Therefore, zi; is a directing word of A', 
and thus L C D(A')- Finally, we have to prove that if qr is a directing word 
of A', then q G L. If q is a directing word of A', then S'(b,q) = 6 is valid for 
every b G A/F where 6 is a fixed element of A/F. Then, 6'{F,q) = 6, and 
thus, b = F since (S'(F, q) = F. This yields that q G L, and hence, D(A') C L. 
Consequently, L = D(A'), and thus, L G Cd. 

Corollary 1. £d is closed under union. 

Now, by Lemma 1 and Proposition 1, we can prove the following state- 
ment. 

Proposition 2. £cnd(2) = ^cnd(3) = ^cnd(i) C £nd(2) = 

and £cnd(i) C £nd(3) = -^d- 



Proof. Obviously, £d Q ^cnb( 2 ) • Now, let L G £cnd( 2 ) arbitrary. Then, 
L ^ 0 and L is regular, furthermore, by Lemma 1, X*LX* = L, and hence, 
by Proposition 1, L € £d- This implies the validity of the first equality. The 
second equality can be proved in a similar way. Regarding the third equality, 
it is obvious again that Cd C £cnd(i) ■ Now, let L G £cnd(i) 0£nd(2) 

be arbitrary. By Lemma 1, X*L = L and LX* = L, and hence X*LX* = L. 
On the other hand, L ^ 0 and L is regular, and thus, by Proposition 1, 
L G Cd, which implies the required equality. Finally, we show the validity of 
the last equality. Obviously, Cd Q ^cnd(i) 0£nd(3)- Now, let L G £cnd(i) H 
^ND( 3 )- Since L G £cnd(i)j by Lemma 1, X*L = L. Since L G £nd( 3 )j there 
is an n.d. automaton A such that L = D 3 (A). Now, we show that X*L = L 
implies that A is a complete n.d. automaton. Contrary, let us suppose that 
= 0 for some state a' and input sign x of A. Furthermore, let w be an 
arbitrary D3-directing word of A. Then, by X*L = L, xw must be also a 
D3-directing word which is a contradiction since a'xw'^ = 0. Consequently, 
A is a c.n.d. automaton, and thus, L G Ccnd( 3 ) = Cd- 
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For the remaining 5 classes we have the proper inclusions presented in 
Figure 1. 

To show that all of the inclusions above are proper, we give examples. 

1. £d C >Ccnd(i) • Let us consider the n.d. automaton Ai = ({1, 2}, {x, y}) 
where = {(1, 1), (1, 2), (2, 1), (2, 2)} and j/-^‘ = {{1, 2), (2, 2)}. Then, Ai 
is a Dl-directable complete n.d. automaton, and y is a Dl-directing word of 
A\. Now, let us suppose that Di(.Ai) E £d- Then, yp £ Di{Ai) must hold, 
for every p £ X*. On the other hand, obviously yx ^ Di(^i) which is a 
contradiction. Therefore, Di(^i) ^ £d- 







CND(2) 




CND(3) 



Figure 1. 

2. £cnd(i) C £nd(i)- Let the n.d. automaton A2 = ({1, 2}, {x,y}) be 
defined by = {(1^ 2), (2, 2)} and = ^^ 2 , 1)}. Then, x is a Dl-directing 
word of A 2 - Assume that Di(^2) ^ ^cnd(i)- In this case. Lemma 1 implies 
that yx £ T>i{A2) which is a contradiction. Thus, Di(^2) 0 ^cnd(i)- 

3. £d C £nd( 2)- Let us define the n.d. automaton As = ({1,2}, {x,y}) 
by x-^3 _ {(1,1), (1,2), (2,1), (2, 2)} and ^ {(1,1)}. Now, x is a D2- 
directing word. Suppose that 02(^43 ) E £d- Then, px £ 02(^43) must hold 
for every p £ X* which is a contradiction since yx ^ 02(^3). Consequently, 
02(^3) i ^D- 

4. £d C £nd( 3)- Let us consider the n.d. automaton A^ — ({1, 2}, {x,y}) 
where x*^ = {(1,2), (2, 1), (2,2)} and y^ = {(1,1)}. Obviously, x is a D3- 
directing word of A\, while yx is not a D3-directing word of A\. Using these 
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two observations and Lemma 1, we obtain that the inclusion considered is 
proper. 

In the following statement, we give a description of £cnd(i) based on the 
deterministic directable automata. Namely, if we consider some sublanguages 
of the languages belonging to £d? then we can get all of the languages from 
£cnd(i)- To present this description, we need some preparation. Let A = 
(A, X, S) be a deterministic directable automaton. Then, a state a' of A is 
called a receiving state of A if S{A, w) — {a'} for some directing word w of A 
where S{A,w) = {aw : a e A}. Let us denote by Ra the set of all receiving 
states of A. Obviously, Ra 0- Then, the following fact is obvious. 

Fact. Let 0 7 ^^ Q C Ra- Then, 

Dq[A) = [w : w £ X* Sz S{A,w) = [a'} for some a' £ Q} 
is regular, Dq(>I) C D{A), and in particular, Drj^(A) = D{A). 

Now, let us denote by £gd the class of languages L C X* such that 
L = Dq(A) for some deterministic directable automaton A and a nonempty 
subset Q of the set of the receiving states of A. Then, for the class £cnd(i) j 
we have the following description. 

Proposition 3. £cnd(i) = ^gd- 

Proof. To prove the inclusion £cnd(i) Q let L £ £cnd(i)- Then, there 
exists a complete Dl-directable n.d. automaton A = (A,X) such that L = 
Di(^). Let us construct now the deterministic automaton 13 = {B,X,S) 
where B = [Ap^ : p £ X*}, and S{C,x) = for every C £ B and x £ X. 
Obviously, 5 is a directable automaton since 6{C, w) = 6(C' ,w) = S{A, w) for 
every C^C £ B and w £ L. Let Q = [S{A,w) : w £ L}. Then, L = Dq(B), 
and therefore, L £ Cqd- 

For proving the converse inclusion, let L £ Cqb- Then, there exist a 
deterministic directable automaton A = (A, X, 5) and a nonempty subset Q 
of its receiving states such that L = Dq{A). Let us construct the B = {B^X) 
c.n.d. automaton as follows: 



(1) B = A\J [d^ : a £ Ra \ Q} where ^ A for every a £ Ra \ Q and 

^ if a ^ b. 

(2) For every b £ B and x £ X, let 



bx^ = 



f{6{b,x)} 

{5(a,x)} 
. {u,u^} 



if b £ A k 6{b,x) ^ Ra\Q, 
if b £ A k s(b, x) = u £ Ra \ Q, 
if b = k S{a, x) ^ Ra \ Q-> 

if b = k S{a, x) = u £ Ra \ Q- 
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Now, it is easy to prove by induction on the length of words that, for 
every 6 E and p € 



bp^ = 



{(5(a,p)} 



if b£Ak S{b,p)^RA\Q, 
if b G A & S{b,p) = u G Ra \ Qj 
if b = Sz S{a,p) ^ R\Q, 
if b = & S{b,p) = u G Ra \ Q- 



Now, we show that L = Di(B). First, let us suppose that w G L. Since 
L = Dq{A), there exists a state u G Q such that S{a,w) = u, for all a G A. 
Then, by the above observation, aw^ = {u}, for all a G A. On the other hand, 
by the above observation again, = {5(a, zi;)} = {u}. Consequently, 

bw^ = {u}, for ail b G B, and thus w G Di(B). 

Conversely, let w gDi (B). Then, there is a state u G B such that bw^ = 
{u} is valid, for all b G B. By the observation above, if u E Ra \ Q or u = 
for some v G Ra\Q, then \uw^\ = 2. Consequently, u G (A\R^)UQ. On the 
other hand, bw^ = {w}, for all b G B, implies that S(a, w) = u, for all a G A, 
which means that u is a, receiving state of A, and hence, u G Ra- From the 
two relations concerning w, it follows that u G Q. Therefore, we have that 
u G Q and 6(A,w) = {u}, which means that w G Dq(^) = L. Consequently, 
L = Di{B). 



It is known (see [6] or [8]) that the regular languages satisfying = L 
are the regular ultimate definite languages. The next observation shows that 
^CND(i) is a proper subclass of the class of the regular ultimate definite 
languages. 

Proposition 4. There exists a nonempty regular language L satisfying 
X* L =■ L such that L ^ ^cnd(i) • 

Proof. Let X = {^r, 2/} and L = U X*yxy. Obviously, T ^ 0, moreover, 
L is regular and X*L = L. Now, let us suppose that L G £cnd(i)- Then, 
there is a c.n.d. automaton A = {A^X) with L = Di(^). Since x^yxy G L, 
Ax-^ = {c} and A{yxy)'^ = {d} for some states c,d G A. Let us observe that 
{c}x‘^ = {c} and = {d}. Now, consider the word yxxy. A{yxxy)'^ = 

A({yx)'^x'^)y'^ = {c}y^ = {d}. Consequently, yxxy G Di(.4) == L, which is 
a contradiction. 



Lemma 2. Let L G £cnd(i) o.nd y G X. Then, Ly G £cnd(i)* 

Proof By Proposition 3, from L G £cnd(i) it follows that L G £qd- Hence, 
there exists a deterministic directable automata A = {A, X, 6) and a non- 
empty set Q C Ra such that L = Dq(^). Now, let us define the automaton 
B = {B,X,S') as follows. 

(1) B = AU {ay : a G Q} where ay ^ A and Uj, ^ if a ^ a'. 
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(2) 5'{a,y) = ay, for all a E Q, 

6\a,x) = 6{a,x), for all a E Q and x £ X \ { 2 /}, 

S'{ay,x) = 5{a,yx), for every a E Q and x £ X, 

S' {a, x) = S{a, x), for every a G A\Q and x £ X. 

Obviously, B is a deterministic directable automaton. Let P = {ay : a £ Q}. 
Then, P C and Ly = Dp(B), and thus, by Proposition 3, Ly £ £cnd(i)- 

Proposition 5. Let w £ X* and L = X^^w. Then, L £ £cnd(i)* 

Proof. Let A = ({1},X) be the c.n.d. automaton with x"^ = {(1, 1)}, for all 
X £ X. Obviously, X* = L>i(A) £ £cnd(i)- By a consecutive application of 
Lemma 2, one can obtain that X*w £ £cnd(i)- 

Corollary 2. £cnd(i) *5 not closed under union. 

Proof. By Proposition 5, X*x,X*yxy £ £cnd(i)- On the other hand, by the 
proof of Proposition 4, we have that the union of these two languages is not 
included in £cnd(i)- 

Unlike the case of £cnd(i)> ^ ^nd( 2 ) does not hold in general. To 
show this, for every x £ X, let us denote by x~^ the language [x* : t = 
1,2,.. .}. Then, the following assertion is valid. 

Proposition 6. Let x £ X and w £ X~^ \ (U{x"^ : x £ X}), furthermore, let 
n be an arbitrarily fixed positive integer. Then, x^X* £ £nd( 2 ) 0 ,'^d wX* ^ 
^ND(2) • 

Proof. Let us consider the automaton A = (A,X) defined by 

A = {l,2,...,n,n-fl}, 

= {(1, 2), (2,3), . . . , (n,n + 1), (n + 1, n + 1)}, 

= {(n + l,n + 1)}, for all j/ € X \ {x}. 

It is easy to see that = D 2 (^), i.e., x^X* £ £nd( 2 )- 

Let w £ X+ \ (U{ir+ : x £ X}). Then, w = yw' = W\ZW 2 for some 
y ^ z £ X and a nonempty word wi £ X*. Now, we show that wX* 0 >Cnd( 2 ) • 
Contrary, let us suppose that wX* £ £nd( 2 )- Then, wX* = 02 (B) for some 
n.d. automaton B = (B,X). First, we prove that bwf 0 is valid, for all 
6 E B. In the opposite case, b'wf = 0 for some b' £ B. Then, bw^ = 0, 
for all b £ B, since w £ D 2 (B). This implies that b{zw)^ = 0, for every 
b £ B, and therefore, zw £ D 2 (B) which is a contradiction since the first 
letter of any words fi:om lyX* is y and y ^ z. Consequently, bwf ^ 0, for 
all 6 E B. Now, let 6 E B be arbitrary and let us consider b{wiw)^. Since 
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bwf ^ 0 and w E D2(B), b{wiw)^ = bw^. Since b is arbitrary, this means that 
wiw € D2(B), and hence, wiw £ wX*. On the other hand, wiw = wiyw* and 
w = W1ZW2, where y ^ z. This results in wiw ^ wX* which is a contradiction. 
Consequently, wX* ^ >Cnd(2)- 

It is known (see [6]) that the regular languages satisfying LX* = L are 
the regular reverse ultimate definite languages. The next observation shows 
that £nd(2) is a proper subclass of the class of the regular reverse ultimate 
definite languages. 

Corollary 3. There exists a nonempty regular language L satisfying LX* = 
L such that L ^ '^nd(2) • 

Proof. Let w £ : x £ X}). Then, by Proposition 6, wX* ^ £nd(2) 

though wX* is a nonempty regular language satisfying {wX*)X* = wX*. 

Corollary 4. £nd(2) "^ot closed under union. 

Proof. Let x,y £ X with x ^ y. Then, by Proposition 6, x“^X*, y^^X* £ 
>Cnd(2)- Now, we show that the language L = x^X* U y^^X* is not included 
in £nd(2) • Contrary, let us suppose that L £ £nd(2) • Then, there exists an 
n.d. automaton A = {A,X) such that L = D2(^). Since x‘^ £ L, = 

a'{xx)'^, for every pair a, a' £ A. Therefore, if a(xa:)*^ = 0 for some a £ A, 
then a{xx)'^ = 0 , for all a G But in this case, a{yxx)'^ = 0 , for all a £ A. 
This yields yxx £ L which is a contradiction. Therefore, a(xx)'^ 7^ 0 , for all 
a£ A, which implies that ax-^ / 0 , for all a £ A. From the latter observation 
it follows that a{xyy)'^ = o,{yy)’^ = a^{yy)'^ = a^xyy)"^ is valid, for every 
pair a, a' £ A. This means that xyy £ L which is a contradiction. Therefore, 

P ^ ^ND( 2 ) • 

Regarding the intersection, the classes of languages considered are closed 
under nonempty intersection as the following assertion shows. 

Proposition 7. Each of the classes of languages considerd is closed under 
nonempty intersection. 

Proof The validity of the statement for £d follows from Proposition 1. Now, 
let us consider £nd( 2)* Let Li,L2 £ £nd(2)j ^tnd let us suppose that Li fl 
L2 ^ 0 . Then, there are n.d. automata A and B such that Li = D2(^) and 
L2 = T>2{B). Let us construct the direct product Ax B. This direct product 
is also an n.d. automaton. Now, it is easy to check that Li H L2 = D2(^ x B) 
which yields that £nd( 2) is closed under the nonempty intersection. As far 
as the remaining classes are concerned, one can prove the validity of the 
statement in the same way as above. 
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Synchronized Shuffle and Regular Languages 



Michel Latteux and Yves Roos 



Summary. New representation results for three families of regular languages are 
stated, using a special kind of shuffle operation, namely the synchronized shuffle. 
First, it is proved that the family of regular star languages is the smallest family 
containing the language (a + 6c)* and closed under synchronized shuffle and length 
preserving morphism. The second representation result states that the family of e- 
free regular languages is the smallest family containing the language (a + 6c)*d and 
closed under synchronized shuffle, union and length preserving morphism. At last, it 
is proved that Reg is the smallest family containing the two languages (a + 66)* and 
a+(a6)“*", closed under synchronized shuffle, union and length preserving morphism. 

1 Introduction 

Finite automata are very popular objects in Computer Science and are now 
used in several other scientific domains. The family of regular languages, 
called Reg, is a basic class in Chomsky hierarchy. The significance of this 
family is strengthened by a great number of its nice characterizations. Some 
of these characterizations state that Reg is equal to the closure of a small 
family under a given set of operations. Besides the famous Kleene’s theorem, 
one can recall the morphic compositional representation of regular languages 
establishing that Reg is the closure of the family reduced to the single lan- 
guage a* 6 under morphism and inverse morphism. This nice result was proved 
in 1982 by Culik, Fich and Salomaa [1]. After the existence of such a morphic 
representation had been proved there followed a sequence of papers improving 
and generalizing this theorem (see [4], [6] and [9]). 

We shall prove here similar characterizations of Reg involving a special 
kind of shuffle. The shuffle operation appears as a fundamental operation in 
the theory of concurrency and admits several variations such as literal shuffle, 
insertion, etc.. Recently, Mateescu, Rozenberg and Salomaa introduced the 
notion of shuffle on trajectories [7] that provides a way to find again the 
most of these variations. Synchronized shuffle was introduced in [2] by De 
Simone under the name of produit de mixage(see also [5], where a closely 
related operation was introduced). This special shuffle can bee seen as a 
shuffle with rendez-vous and is very useful in modelling the behaviours of 
parallel processes (see [3] and [8]). This powerful operation is quite amazing. 
For example, contrary to what was stated in [2], it is not associative. 
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The study of synchronized shuffle in connection with union and length 
preserving morphism provides us to obtain new representation results for 
three families of regular languages. 

First, we prove that the family of regular star languages is the smallest 
family containing the language (a+bc)* and closed under synchronized shuffle 
and length preserving morphism. Our second representation result states that 
the family of e-free regular languages is the smallest family containing the 
language (a + 6c)*d and closed under synchronized shuffle, union and length 
preserving morphism. At last, we prove that Reg is the smallest family con- 
taining the two languages (a + bb)* and a-f (a6)“^, closed under synchronized 
shuffle, union and length preserving morphism. 



2 Preliminaries 

2.1 Notations 

Let X be an alphabet. For any word it; G X*, we shall denote by |it;| the 
length of the word w and for any letter a € X, we shall denote by \w\a the 
number of occurrences of the letter a that appear in the word w. 

For any language L C X*, we denote by a(L), the alphabet of L that is : 
a(L) = {x £ X \ 3u L : \u\x > 0} 

For example we get : a{a + ab) = a{o? + b^) = {a,b}, a(0) = a{e) = 0. 

We denote by 77x,y the projection over the sub-alphabet Y, i.e. the ho- 
momorphism from X* to Y* defined by: 

Vx G X, if X G y then IIx,y(x) = x , else i7x,y(x) = e 

When there is no ambiguity, we shall use notation iZy instead of IIx.y- 
u Lu V is the shuffle of the two words u and v that is 

U in V = {uiViU2V2‘--UnVn \ ^ ^ = UiU2--Un^V = ViV2.--Vn} 

This definition is extended over languages by : 

VLi,L‘2 C X*,Li lu L 2 — U U)\ LU W2 

tui 6 Li 
^2 € 1^2 

When there is no ambiguity, for a language reduced to a single word, we 
shall write this language u rather than {u}. 
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2.2 Synchronized shuffle 

Definition 1. [2] Let X be an alphabet and L\ , L 2 be two languages in- 
cluded in X*. The synchronized shuffle of Li and L 2 , denoted by L\ m L 2 is 
defined by : 

Li m Z/2 = {ii; € (a(Li) U a(L2))* | E {1,2}} 

Examples : 

1. bab m cac = {bcabc^ bcacb, cbabc^ cbacb} 

2. bac m cab = 0 

3. {a, abb} m aft = 0 and a m aft = aft 

4. (a m {a, ft}) m aft = a m ab = ab 

5. a m ({a, ft} m aft) = a m 0 = 0 

Example 3 shows that it generally does not hold that 

Li m L 2 = IJ Wi m W ‘2 

u»i € Lj 
102 € 1^2 

Examples 4 and 5 show that synchronized shuffle is not an associative oper- 
ation. 

The following properties and proposition easily come from definition : 

• the synchronized shuffle is commutative :Li m L 2 = L 2 m Li 

• L m e = L 

• L m i/j = i/j 

• if a(Li) = a(L 2 ) then Li m L 2 = Li fl L 2 

• if a{Li) n a(L 2 ) = 0 then Li m L 2 = Li lu L 2 

• (L* m L 2 ) = (L* m L 2 )* 

• (£■ E L\ m Z/ 2 ) s — 7 ^ (£■ E Li n Z/ 2 ) 

• if Li and L 2 are e-free languages then for any letter a :(a E Li m L 2 ) 4=^ 
(a E L\ n -L 2 ) 

Proposition 1. Let L\ and L 2 be languages with Xi = a{Li) for i E {1,2}. 
Then the following equality holds : 

Li m La = (Li m (X 2 \ Xi)*) f|(L 2 m (Xi \ Xa)*) 

2.3 synchronized shuffle and associativity 

As we have seen in examples, the synchronized shuffle is not associative. 
We shall give now a sufficient condition for the synchronized shuffle to be 
associative over a (m -closed) family of language. For every family of language 
£, we denote by (£)m the least family of languages containing C and closed 
under m . First we can state : 
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Proposition 2. Let C be a family of languages. If for every languages L\, 
L2 € {C)m) equality a{Li m L2) = ol{Li) U q:(L2) holds then m is 
associative over {C)m • 

Proof Let Li,L2,L^ € (>C)m then, from definition, we have 
W G (Li m L2) m Ls (^a(Li m La)^ ^ ^ ^2) A {IIc,(L 3 )M ^ 

Since o:(Li m L2) = a(Li) U 0(^2), we get : 



W G (Li m L2) m L3 (iIa(LiUL2)(^) ^ ^ ^2) A {IIa{L3){u) G L3) 

Moreover 77 c,(l.)(t/) = na(Li){na{Li)ua{L2)M) for i G {1, 2}, hence we have : 

U G (Li m L2) n-1 L3 



(^q(Li)(^) ^ ^1) A (iIcK(L2)(^) ^ ^2) A {IIa{L 3 )i'^) ^ ^3) 
and m is associative over (£)m • 

Definition 2. Let L be a language. A language R is said to be L-compatible 
if the two following conditions are satisfied : 

1 . n^^n)(L)CR 

2 . a{R) C a{L) 

A family of languages C is said to be L-compatible if every language in £ 
is L-compatible. 

Lemma 1. Let L be a language and i?i,i ?2 be two languages which are L- 
compatible. Then : 

1 . a{Ri m R2) =■ (x{Ri) U o:(i?2)- 

2 . Ri m R2 is L-compatible. 

Proof. From 1 of definition 2, LTct(i?i)ua(H2)(^) Q Ri ^ R‘i and from 2 of 
definition 2, we also have 



c»:(^a(/ei)ua(i?2)(^)) = ot{Ri) U a{R 2 ) 

thus a(i?i) U a{R2) C a{R\ m R2). From definition of synchronized shuffle, 
a(JRi m R2) C a{Ri) U a(i?2) then a{Ri m R2) = a(i?i) U a(i?2) C a(L). It 
follows that 



na{Ri m i?2)(^) = ^a(i?i)Ua(/?2)(^) Q Rl ^ R ‘2 

hence Ri m R2 is L-compatible. 

Then, by induction and from proposition 2 it directly follows : 

Proposition 3. Let L be a family of languages. If there exists a language L 
such that C is L-compatible, then (£)m Is L-compatible and m is associative 
over (£)nn 

In the following, we shall omit use of parenthesis in expressions involving 
synchronized shuffle over such families of languages. 
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3 A new characterization of Reg 

Here, we shall prove characterization results involving synchronized shuffle. 
These results concern several families of regular languages. First, let us con- 
sider Fine, the family of e-free finite languages. 

Proposition 4 . The family Fine is the smallest family containing the lan- 
guage a + ah and closed under union, length preserving morphisms and syn- 
chronized shuffle. 

Proof. Let C be the smallest family containing the language a-\-ab and closed 
under the three above operations. Clearly, C C Fine • Conversely, since union 
is allowed it is sufficient to prove that : 

1. 0 is in £ 

2 . for every alphabet X = {xi,X2, . . . ,Xn},n > 0 , the language reduced to 
the single word X\X2 • --Xn is in C. 

For 1 , we get 0 = (a + ab) m (6 -h ba). For 2 , we make an induction on n. If 
n < 3 , we get a = [(a4-a6)+(fe4-fec)] m [(a-l-ac)-l-(c-l-cfe)] and ab = (a-\-ab) m b. 
If n > 3 , it is easily seen that xiX2 . ..Xn = x\X2 . . . Xn-i xi- --Xn- 

We shall prove a similar result for Reg^, the family of regular star lan- 
guages. The starting language is (a 4- be)* and we use only length preserving 
morphisms and synchronized shuffle. First, we consider monoids generated 
by special finite languages. 

Definition 3 . A marked language F is a finite language such that : 

E F,Vx E a(F), (\u\x > 0 A \v\x > 0 ) => {u = v A \u\x = 1 ) 

Next, we show that if F is a marked language, then F* can be obtained 
from languages of the type (a 4- be)* using synchronized shuffle. 

Definition 4 . Two languages L\ and L2 are said equivalent if there is a 
length preserving morphism which maps bijectively L\ onto L2. 

Lemma 2 . For every marked language F , F* can be obtained from languages 
equivalent to {a 4- be)* using synchronized shuffle. 

Proof. Let C be the family of languages which can be obtained by synchro- 
nized shuffle from languages equivalent to (a 4- 6c)*. The languages a* = 
(a 4 - be)* m (a 4 - eb)* and {ab + ed)* = (a 4 - ed)* m (c -f ab)* belong to £. 
Hence, (a 4- 6)* = a* m b* , (ab)* = (ab-\-cd)* m {ab-\-dc)*, e = {oh)* m {ba)* 
are in C*. 

Let us consider now a marked language F = u-\-v with |i;| > \u\ > 0 . From 
the above equalities, if |t^| <2 then F* E C. Assume that v = X1X2 . . .Xk with 
A: > 3 . The languages Li = {u-\-xi.. .xa;_i)*, L2 = (u4-X2 . . .Xk)* and L3 = 
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(u -f xiXkY are F-compatible. Prom the equality (u -h v)* = Li m L2 m L3, 
we get, by induction, that {u + v)* e C. At last, F = m -\- U2 -h Un 
is a marked language with n > 3, the languages Ri = (ui Un-i)*, 

Ri — (^2 -h . . . + UnY and Rz = (ui + UnY are F-compatible. Once again, 
the equality F* = m R2 m F3 implies by induction that F* G C. 

We can now get easily our characterization result for the family Reg^ . 

Proposition 5. The family of regular star languages is the smallest family 
containing the language (a -h bcY dfid closed under length preserving mor- 
phisms and synchronized shuffle. 

Proof. Clearly, Reg^ is closed under the above operations. For the reverse in- 
clusion, let R* be a regular language. It is known (see [4], [6]) that there 
exist two finite languages Fi and F2 and a length preserving morphism 
such that R* = g{F* n F2). One can assume that a(Fi) = a(F2). Hence, 
R* = g{Fi m F2). Moreover, Fi and F2 are the image by a length preserving 
morphism of marked languages. Thus, lemma 2 implies the result. 

We are now able to state our first characterization of Reg. 

Proposition 6. The family of regular languages Reg is the smallest family 
containing the languages (a+6c)* and a, closed under union, length preserving 
morphisms and synchronized shuffle. 

Proof. Let C be the smallest family containing the languages (a 4- bcY and 
a, closed under the three above operations. Clearly C is included in Reg. For 
the reverse inclusion, let us consider a language R G Reg. From proposition 
5, we know that e is in £, so we may suppose that e ^ R since union is 
allowed. Hence one may assume, without loss of generality, that R C A* A' 
where A and A' are disjoint alphabets. Then R = R* m A'. From proposition 
5, R* can be obtained from (a-hfcc)* using length preserving morphisms and 
synchronized shuffle. On the other hand. A' can be obtained from a using 
union and length preserving morphisms. 

We can observe that it is not possible to enunciate a similar result with 
a single generator. Indeed, since e G Lim L2 if and only if £ G Li D L2, we 
need to start from a family containing at least two languages Li and L2 with 
s G Li and e ^ L2. The following result concerns the family of e-free regular 
languages, that is regular languages which do not contain the empty word. 
For this family, it is possible to start from a single generator. 

Proposition 7. The family of regular e-free languages Reg^ is the smallest 
family containing the language (a + bcYd and closed under union, length 
preserving morphisms and synchronized shuffle. 
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Proof. Let C be the smallest family containing the language (a -\- he)* d and 
closed under the three above operations. Clearly C is included in Reg^. For 
the reverse inclusion, let us consider a language R E Reg^. Without loss of 
generality, one may assume that R is a finite union of languages in the form 
Kd with K G Reg and d a letter not in a{K). It remains to prove that such 
a language Kd is in C. This can be done by induction over the construction 
of K with respect to proposition 6. 

If K = {a + be)* then Kd is in C. If K = a, observe first that d = 
(a -h ab)*d m (6 -h ba)*d is in C. Moreover a*d, which can be obtained fi:om 
(a be)* d using a length preserving morphism, is in C. Then we get ad = 
a m a*d belongs to C. 

Now, 

• if K = h(K') for some K' G C and some length preserving morphism h, 
we may suppose that d ^ a(RT') then K'd G C which implies Kd G C. 

• if K = Ki-\- K ‘2 with Ki G C for i = 1, 2 then Kd = Kid + K 2 d G C. 

• if K = Ki m K 2 with Ki G C for i = 1, 2 . We may suppose that d ^ a{Ki) 
for z = 1, 2 then Kd = Kid m K 2 d G C. 

4 Binary generators 

The single generator for the family of e-free regular languages Reg^ used in 
proposition 7 is built over a four letter alphabet. The following proposition 
shows that it is possible to start from a language defined over a three letter 
alphabet : 

Proposition 8. The family of regular e-free languages Reg^ is the smallest 
family containing the language (a + be)*b and closed under union, length 
preserving morphisms and synchronized shuffle. 

Proof. Let C be the smallest family containing the language (a be)* b and 
closed under union, length preserving morphisms and synchronized shuffle. 
Prom proposition 7, we have to prove that (a -\-bc)*d is in C. Observe first 
that 6 = (a 4- bc)*b m (c + ba)*b is in £. Then a*b = (a + be)*b m b and 
(a 4- bc)*bd = (a + hc)*b m b*d are in C. It follows that (a + bc)*bc is also 
in C. Let us consider now the length preserving morphism h defined by : 
h(a) = a, h{b) — h{d) — b and h{c) = c. We get 

(a -1- be)* be m h{{a -f cb)*c m d) = a*{be)~^ G C 

then a*{bh)^ G C and a* 6+ = a*h 4- a* (66)+ + /i(a*(66)+ m a*d) is in C. 
Moreover a+6* = a+ -h (a* 6+ m a+) is also in C since, clearly, a+ G £. Now, 
with the length preserving morphism g defined by : g{a) = g(e) = a,g{b) = 
6, g(c) = e and g(d) = d, we obtain 

(a 4- be)*bca*d = g{{a + bc)*bc m c+e* m e*d m c*c?) G C 

At last, we get (a 4- be)*d = a*d + (a -i- bc)*bca*d G C. 
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We have seen that the family of (s-free) regular languages can be obtained 
from languages whose cardinality alphabet is less or equal to three. A natural 
question is whether it is possible to start from binary languages which are 
languages built over two letter alphabets. In a first time, we shall establish a 
negative answer for the family of star regular languages. 

Definition 5. A star language L satisfies the (P) property if there exist 
three distinct letters a, 6, c such that the words a and be are in L, but the 
word bac is not in L. 

Lemma 3. Let S be a set of star languages and C be the smallest family 
of languages, containing S and closed under length preserving morphism and 
synchronized shuffle. If C contains a language which satisfies the (P) property 
then so do S. 

Proof First, it is clear that £ contains only star languages. We shall prove 
this lemma by induction. Let L be a language in £ with a,bc G L and bac ^ L. 

If there exist some language V G C and some length preserving morphism 
h such that L = h{V) then it is obvious that V satisfies the (P) property. 

Let us suppose now that L = Li m L 2 such that neither L\ nor L 2 satisfies 
(P). We shall show that it will lead to the following contradiction bac G L. 
For i = 1,2, we denote ai = a{Li) fl {a, 6, c}. Observe that bac G L and 
only if Haiibac) G Li for i = 1,2. Moreover, since L is a star language, the 
words abc and bca are in L. Now, for z = 1, 2 : 

• if b, c}, we get 77^^ (bac) G II ai {abc 4- bca) C Li 

• if o;i = {a, 6, c}, then a G Li and be G Li. Since Li does not satisfy (P), 
it follows that IIai{bac) = bac G Li. 

The contradiction bac G L implies that Li satisfies (P) or Lz satisfies (P). 

Since the language (a + 6c)* satisfies the (P) property, we can state, as a 
corollary of the above lemma : 

Proposition 9. The family of regular star languages can not be obtained as 
the closure of a set of binary languages under length preserving morphism 
and synchronized shuffle. 

For the family of (c-free) regular languages, the answer is positive. In 
order to establish this result, we shall first prove the following lemma : 

Lemma 4. Let £ be a family of languages containing the language a"^b* and 
closed under length preserving morphism and synchronized shuffle, then £ is 
closed under product. 

Proof. Let Li and Lz be two languages of £. We may suppose that a\ = 
q:(Li) and az = ol{L 2 ) are disjoint. The family {x*?/* \ x G a\,y G az} 
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is clearly LiL 2 -compatible, then synchronized shuffle is associative over this 
family and 

m = 0L\0L2 ^ ^ 

a € c*! 
y € OC2 

Then L 1 L 2 = {Li m L 2 ) m is in C. 

We are now able to enunciate our la;St proposition which states that Reg, 
the family of regular languages can be obtained from binary generators. 

Proposition 10. The family of regular languages Reg is the smallest family 
containing the languages a + {ab)~^ and (a H- bb)*, and closed under union, 
length preserving morphism and synchronized shuffle. 

Proof Let C be the smallest family containing the languages a + (ab)~^ and 
(a bb)*, closed under the three above operations. Clearly C is included in 
Reg. 

For the reverse inclusion, observe first that 

(a + bb)* m (a 4- (afc)*^) = a E C 

It remains to prove that (a 4- 6c)* G C. Let h be the length preserving 
morphism defined by h{a) = h{b) = a, we get h{{a 4- bb)*) = a* £ C 
and h{a* m b) = a'^ G C. It follows that (a + (ab)'^) m b = ab G C and 
(a 4- (a6)+) m 6+ = (a6)+ G C. 

Now, if g is the length preserving morphism defined from by g{a) = g(d) = 
g{e) = a, g{b) = b and g{c) = c, we get 

g{{cic)'^ m bd) m ^((ca)'*' m be) = 6(ac)“^a G C 

Then b{aa)^a in £, g{b{aa)^a m bd) = 6(aa)"*"aa G C and g{ba m ad) = 
baa G C so : 



ba* = 6 4- 6a 4- baa 4- b{aa)^a 4- 6(aa)"^aa G £ 

In a same way we can prove a*b G C. It follows that a*b m be* = a*bc* is in 
C then a*b^ is in £. Now, since a*b^ 4- a* = a*b* G C and from lemma 4, we 
get that C is closed under product. 

We are now able to obtain (a 4- 6c)* : 

(((a 4- 66)* m (a 4- cc)*) m (6c)'^) -f a* = (a + 6c6c)* G C 

The family C is closed under product then we also have bc(a 4- bcbc)*bc G C. 
Then 



Li = (a -h bcbcY m bc{d + bcbc)*bc = a*6c(d*6ca*6c)*d*6ca* € C 
and, since product is allowed L 2 = a*bcLi G £. We finally get that 
g{Li) + g{L 2 ) + a*bca* -h a* = (a + 6c)* G C 
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Synchronization Expressions: 

Characterization Results and Implementation 
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Summary. Synchronization expressions are defined as restricted regular expres- 
sions that specify synchronization constraints between parallel processes and their 
semantics is defined using the synchronization languages. In this paper we survey 
results on synchronization languages, in particular, various approaches to obtain a 
characterization of this language family using closure under a set of rewriting rules. 
Also, we discuss the use and implementation of synchronization expressions in a 
programming language designed for a parallel or distributed computing environ- 
ment. 



1 Introduction 

Synchronization expressions (SE) were originally introduced in the parallel 
programming language ParC [8] as high-level constructs for specifying syn- 
chronization constraints between parallel processes. Synchronization requests 
are specified as expressions of tags for statements that are to be constrained 
by the synchronization requirements. 

The semantics of SEs is defined using synchronization languages. The 
corresponding synchronization language (SL) describes intuitively how an 
SE is implemented in the parallel programming language ParC. Using this 
formalism, relations such as equivalence and inclusion between SEs can be 
easily understood and tested. Thus the synchronization languages provide 
a systematic approach for the implementation and simplification of SEs in 
parallel or distributed programming languages. 

A synchronization language can be seen as the set of correct executions 
(as controlled by the SE) of a distributed application where each action is 
split into two atomic parts, its start and termination. It can be argued [9,10] 
that synchronization languages are a more suitable semantic model for SEs 
than traditional models such as traces, Petri nets, or process algebra [11,19]. 
An important feature of the semantics defined by synchronization languages 
is that it splits each action into two parts, the start and the termination. 
For instance, in trace semantics [7] parallel execution is interpreted as a || 
b = ab ba, but in order to control the execution of parallel processes it is 
necessary to distinguish sequences like ab and ba from real parallel execution. 
By having the start and termination of an action as separate instantaneous 
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parts such distinctions can be made. Furthermore, synchronization languages 
can specify that certain occurrences of given processes are parallel whereas 
other occurrences need to be executed in a specified sequential order. The 
relationship of and the differences between synchronization languages and 
the other formalisms used in concurrency will be discussed further in [3]. 

It was shown in [9,10] that synchronization languages are closed under 
certain naturally defined rewriting rules and there it was also conjectured 
that synchronization languages would consist of the subset of regular lan- 
guages closed under these rules and satisfying the so called start-termination 
property. The conjecture was proved for languages expressing synchroniza- 
tion between two distinct actions (i.e, for a two-letter alphabet) by Clerbout, 
Roos and Ryl [5,14] and they showed that the conjecture is false in general. 
Furthermore, [15] establishes a more general negative result showing that 
these languages cannot be characterized by any set of rewriting rules. 

By generalizing the syntactic definition (and modifying the semantics) of 
SEs [18] considered a new definition of synchronization languages. The new 
definition has the advantage that it allows us to eliminate the intuitively less 
well motivated transformations (rewriting rules) describing closure proper- 
ties of the synchronization languages and, furthermore, the new set of rules 
always preserves regularity. This gives us hope to avoid the negative results 
that were obtained for the original definition of synchronization languages. 
The approach allows us to obtain an exact characterization for the finite 
synchronization languages in terms of simple semi-commutation rules [18] 
and a characterization of the images of synchronization languages under st- 
morphisms [16]. However, it is not known whether the family of synchroniza- 
tion languages is closed under st-morphisms and, thus, it remains an open 
question whether the rewriting rules can be used to characterize synchroniza- 
tion languages in general. 

Besides giving rise to interesting questions in formal language theory, syn- 
chronization languages provide us with a systematic method for implementing 
SEs in a parallel programming language such as ParC. This gives one more 
example on the usefulness of the interaction between formal language theory 
and practical computer science. In the final section we discuss the practical 
use and implementation of SEs and SLs. A description of the implementation 
has not been presented previously in published form. Roughly speaking the 
idea is as follows. For an SE, we construct at compile time a finite automaton 
that accepts the prefix language of the corresponding SL. The constructed 
automaton is then used to impose at runtime the synchronization constraints 
specified by the SE. 



2 Synchronization Expressions and Languages 

We assume that the reader is familiar with the basic notions related to regular 
expressions and rewriting systems [1,17,20]. The set of finite words over a 
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finite alphabet E is denoted A is the empty word, and E^ = E* — {A}. 
The catenation of languages Li,L‘2 C E* isLi*L 2 = {w G E* \ G = 
1,2) w = V 1 V 2 } and the catenation of n copies of L C E* is (n >0). Note 
that = {A}. The Kleene star of a language L is L* = U^qL\ 

The shuffle of words u,v G E* is the language u(u,v) C E* consist- 
ing of all words that can be written in the form uiVi • • • UnVn, n > 0, 
where u — v = Ui,Vi G E*, i = The shuf- 

fle operation is extended for languages in the natural way: a;(Li,L 2 ) = 

A string-rewriting system (or Thue system) over A7 is a finite set R of 
rules u V, u,v G E*. The rules of R define the (single step) reduction 
relation C E* x E* as follows. For Wi,W2 G E*, wi ->r W 2 if and only if 
there exists a rule ui U 2 G: R and r,s G E* such that wi = ruis, i = 1,2. 
The reduction relation of R is the reflexive and transitive closure of — and 
it is denoted 

We define synchronization expressions using the extended definition from 
[18]. The operators ->►, ||, |, & and * are called, respectively, the sequencing, 
join, selection, intersection, and repetition operators. Later we give also the 
original more restricted definition that differs only in the use of the join 
operator. 

The intuitive meaning of the operations will be apparent from the seman- 
tic interpretation given below. For easier readability we omit the outermost 
parentheses of an expression. All the four binary operations will be associa- 
tive, so usually also other parentheses may be omitted. 

Let be an alphabet such that each symbol of E denotes a distinct 
process. In order to define the meaning of synchronization expressions, for 
each a G E we associate symbols and cr to denote, respectively, the start 
and the termination of the process. Also, we denote 

Eg = {o^s \ ^ E}, Et = {at \ a G E{. 



The synchronization languages satisfy the intuitively natural condition 
that the start of an occurrence of a process always precedes the termination 
of the same occurrence. The condition is defined formally as follows. For 
a G E, let pa : {Eq U EtY — > {og^atY be the morphism determined by the 
conditions 




X G {ag,at}, 
\ix ^ {ag,at}. 



A word w G (Eg U i^*)* is said to satisfy the start-termination condition (or 
st-condition for short) if for all a G E, Pa{w) G (a«a^)*. If w satisfies the 
st-condition, then w belongs to the shuffle of some languages ((ai)s(ai)^)*, 
• • • , {{ak)8{o>k)tY where a* 7 ^ aj, when i j, i,j = l,...k. 

The set of all words over Eg U Et satisfying the st-condition is denoted 
and subsets of are called st-languages. 

The synchronization expressions over an alphabet E and the languages 
denoted by the expressions are defined inductively as follows. 
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Definition 1. The set of synchronization expressions over an alphabet 
SE(I7), is the smallest subset of (Z’U{0, |, ||, *, (, )})* defined inductively 

by the following rules. The synchronization language denoted by a € SE{U) 
is L{a). 

(i) E U {0} C SE(i7). L{(j)) = 0 and L{a) — {a^a^} when a e E. 

(ii) If ai,a 2 € SE(Z') then (ai 0 : 2 ) ^ SE(i7) and L(o:i — >> 02 ) = L(ai) • 
L(Q2). 

(iii) If ai,a 2 ^ SE(i7) then (ai 1| 02 ) G SE(i7) and 
L{ai II 02 ) = o;(L(ai), L(a2)) H W$. 

(iv) If ai,a 2 ^ SE(i7) then (ai | a 2 ) € SE(i7) and L{ai | 02 ) = U 

L{a2). 

(v) If ai,a 2 € SE(i7) then (ai&a 2 ) £ SE(i7) and L(ai&a 2 ) = L(ai) D 
L(a2). 

(vi) If a E SE{E) then a* E SE(i:) and L(a*) = L(a)". 

For concrete examples of synchronization expressions see the last section 
or [9,10]. When defining the interpretation of the parallelization (join) oper- 
ator in (iii) we use intersection with the set of st-words. If we would allow 
arbitrary shuffles, this would produce words like a^a^a^at that do not have any 
meaningful interpretation. The definition implies that L(a || a) = L{a — > a). 

The original definition of synchronization expressions required that the 
expressions appearing as arguments of the parallelization operator must be 
over disjoint alphabets. The set of restricted synchronization expressions, 
SEr(X'), is defined as in Definition 1 by substituting SEr{E) everywhere for 
SE(E) and modifying the condition (iii) as follows. We denote by alph(a) the 
set of symbols of E appearing in the expression a. (alph(a) can naturally be 
defined inductively following Definition 1.) 

(iii)’ If ai,Q 2 G SEr{E) and alph(ai) D alph(a 2 ) = 0 then (ai || a 2 ) G 
SEr(X') and L(ai || 02 ) = a;(L(ai),L(a 2 )). 

In restricted synchronization expressions, always when a\ || 02 is defined 
each word of u{L{ai),L{a 2 )) satisfies the st-condition so we do not need the 
intersection with the set W^. 

The family of synchronization languages £(SE) (respectively, restricted 
synchoronization languages C{SEr)) consists of all languages L(a) where a E 
SE(i7) (respectively, a E SEr{E)) for some alphabet E. Directly from the 
definition it follows that £(SEr) C £(SE). Furthermore, the inclusion is strict 
since, for instance, the expression (6 a*) || {a* -> c) denotes a language 

that is not in £(SEy) [14,18]. 

3 Partial Characterization Results 

Synchronization languages are regular languages satisfying the st-condition 
and furthermore they are closed under certain types of semi-commutation 
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rules and their extensions. It would be very useful if we could exactly char- 
acterize the synchronization languages in terms of closure under some simple 
rewriting rules. A characterization could yield more efficient algorithms for 
testing the equivalence of SEs since in many cases it would be easy to test 
whether a given finite automaton accepts a language satisfying the given 
semi-commutation properties. 

Definition 2. Let E be an alphabet. We define the rewriting system Rjj C 
{Eg U EtY X {Eg U EfY to consist of the following rules, where a,b G E^ 
a^b: 

(i) agbt “4 btcis, 

(ii) 0 , 3^8 ^ bgCig^ 

(hi) atbt -4 btUt. 

The set is defined to consist of all rules 

(^l)t * * * ‘ * * (1) 

(^l)t * * * * * • (Oi)«5 

where ai , . . . , a* , bi , . . . , 6^ are pairwise distinct elements of E,i,j > 1 . Then 
we denote 

R'x! = Re U Sjj. 

Note that the rules Rx: define a semi- commutation relation on E [4]. By 
symmetry the rules (ii) and (hi) could be written also as two-directional rules. 

If i? is a rewriting system over E and L C E* we denote ^^(L) = {u G 
E* I (3i; G L)v u}. We say that L is closed under Rif L = The 

following results hold. 

Theorem 1. Let L C E* . 

(a) [18] If L is a synchronization language, then L is closed under Rs- 

(b) [9,10] If L is a restricted synchronization language, then L is closed under 

Re- 

The intuitive explanation why restricted synchronization languages are 
closed under rules (1) is that if we have a subword of the form . . . atagbtbs . . . 
(or the given more general form), then all the following pairs of the given 
occurrences of the symbols are parallel: (at, bt), (a«, bs) and (a«, bt), i.e., they 
correspond to symbols occuring in different arguments of the parallelization 
operator. On the other hand, since the different arguments may not contain 
occurrences of the same symbol of E, the only possibility is that the symbols 
Ot^CLg correspond to occurrences in one of the arguments, and bt.bg in the 
other argument. The proof showing closure under the rules of Re is similar 
but simpler. 

It was conjectured [10] that closure under the rewriting system to- 
gether with the st-condition exactly characterizes the restricted synchroniza- 
tion languages. The conjecture was proved in [5] for alphabets corresponding 
to two actions. 
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Theorem 2. [5] A language L C {a^, a*, 6^, 6^}’' is a restricted synchroniza- 
tion language if and only if L is regular j satisfies the st-condition and is closed 
under R^. 

At the same time [5,14] established that the conjecture does not hold in 
general. The counter-example given there is the language 



Li = A*if,^{bs{asat)*Csbtiagat)*ct) 

which is a regular st-language and it can be shown that Li is not a restricted 
synchronization language. The intuitive idea is that restricted synchroniza- 
tion expressions cannot differentiate in L\ between instances of the action 
a that occur during b and during c, respectively. Formally the fact that 
Li ^ C{SEr) is proved by establishing a so called switching property for 
restricted synchronization languages [5,14]. 

One can show that £(SEr) is closed under certain extensions of R^ satis- 
fying nice projection properties onto subalphabets [14,15]. However, the hope 
of obtaining a rewriting characterization for this class was destroyed by the 
following result. 

Theorem 3. [15,6] There does not exist any rewriting system Q such that 
each L £ £(SEr) is closed under Q and each regular st-language closed under 
Q is in C{SEr). 

It may be noted that another undesirable property of the restricted syn- 
chronization languages is that the corresponding rewriting rules (1) do not, 
in general, preserve regularity of st-languages. Consider the language L <2 = 
asbsiatasbtbsYbtat. Using the rule atasbtbg -> btbsatas (and the rules (i)-(iii) 
of Definition 2) every word of L 2 can be rewritten to a word (bsbt)^ (agat)^ , 
m > 1, and this observation easily implies that zi^,^(L 2 ) is not regular. 

Due to the negative result of Theorem 3 it is natural to consider the more 
general definition of SEs given in Definition 1. This allows us to drop the rules 
(1) from the rewriting system describing closure properties of the synchro- 
nization languages. Making use of general properties of semi-commutations 
[4] we can then prove: 

Proposition 1. [18] If L C {UgUUtY is a regular st-language, then 
is also regular. 

Using the more general definition, in the case of finite languages we can 
strengthen the result of Theorem 1(a) into an “if and only if” condition. 

Theorem 4. [18] Assume that L is a finite language over Ug ^ Then 
L £ £(SE) if and only if L is an st-language closed under Rjj. 

Interestingly it turns out that closure under Rjj exactly characterizes 
the images of synchronization languages under functions called st-morphisms 
[15,16]. 
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Definition 3. Let U and Q be alphabets and (p : Z** — > a strictly 

alphabetical morphism (that is, ip{a) E f2 for each a G U). The morphism ip 
is extended in the natural way to a morphism (p : — > (f2gU OtY 

by setting (p{as) — <p{a)s and (p{at) = (p(a)t for each a G E. 

With each strictly alphabetical morphism ip : E* — > i?*, we associate a 
partial function (p called an st-morphism: 

tp= {(u, ifiu)) I u e and <f(u) € 

The family of st-morphisms is denoted #st* 

Note that ip is equal to the relation o (p o (nW|;) and an st- 

morphism is not, strictly speaking, a morphism. For instance, if ip(a) = ip{b) 
then (p{asb8atbt) is not defined. 

Theorem 5. [16] ^st(^(SE)) equals to the family of regular st-languages L C 
E* that are closed under the rewriting system Rjj. 

We conjecture that in general a regular st-language is a synchronization 
language if and only if it is closed under Rjj, i.e., that we could drop the 
finiteness condition from Theorem 4. However, we do not have a proof for this 
conjecture. According to Theorem 5, the conjecture is equivalent to showing 
that 

^st{C{SE)) = £(SE), 

that is, that the family of synchronization languages is closed under st- 
morphisms. In [15] it is shown that #st(^(SE^)) = ^st(^(SE)) which implies 
that the family £(SE^) is not closed under st-morphisms since it is strictly 
included in £(SE). 

4 Implementation of SEs in Concurrent Programming 
Languages 

In this section, we explain how the ideas of SEs and SLs can be used and 
implemented in a programming language that is designed for a parallel or/and 
distributed computing environment. The implementation follows strictly our 
semantic interpretation of SEs using SLs. More specifically, for an SE, we 
construct at compile time a finite automaton that accepts the prefix language 
of the SL denoted by the SE. Then the constructed automaton is used to 
impose the synchronization constraints specified by the SE at runtime. 

In the following, we will use examples written in the ParC concurrent 
programming language [8]. The examples will be explained in detail assum- 
ing that the reader knows the C programming language but not ParC. In 
ParC, (simple or compound) statements are considered as the basic elements 
of synchronization. Statements that are involved in synchronization are rep- 
resented in SEs by statement tags. In other words, the alphabet of an SE is 
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a set of statement tags. This is a design decision specific to ParC and need 
not be true in general. Especially in an object-oriented concurrent program- 
ming language, it may be more appropriate that object operations rather 
than statements should be the basic elements of synchronization specified by 
SEs. However, the implementation principle described below would still be 
applicable as long as a symbol in an SE represents an entire execution of a 
process. 

We provide in the following a simple example to show how a ParC program 
with synchronization expressions is translated into a Sequent C program with 
system calls [13]. We first give the ParC program with a detailed explanation. 
Then we give the generated code in Sequent C with system calls. 

main(){ 

shared int w; 
int r; 
int f 0 ; 
tag a, b; 
restrict (a->b)*; 

pexecf 

{/* */ 

b: : r = w; 

/* */ 

} 

{/* */ 

a: : w = f () ; 

/* */ 

} 

} 

} 

We first look at the pexec statement, i.e., the parallel execution statement. 
This language construct denotes that all statements that are directly un- 
der the scope can be executed in parallel. In the above program, there are 
two compound statements directly under the pexec statement, which can be 
considered as two parallel processes. In the first process, the statement ‘Y = 
w;” is tagged with b, and in the second process, “w = f();” is tagged with 
a. We assume that each of the two statements is in a loop. The restrict 
statement above the pexec statement specifies a synchronization expression 
(a->b)*, meaning that the a and b statements can be executed only in the 
following sequence: a, b, a, b, ..., i.e., the ith instance of b cannot start 
its execution before the ith. instance of a finishes and the (i -f l)th instance of 
a cannot start its execution before the ith instance of b finishes, i > 1. Also 
in the declarations, w is declared as a shared integer variable, which means 
that the two appearances of w in the two parallel processes, respectively, are 
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the same variable. The integer variable r is not shared. So, it has different 
copies in different parallel processes in the scope. And the statement tag a, 
b ; declares that both a and b are statement tags. 

A deterministic finite automaton (DFA) is built according to the SE a->b, 
which accepts the prefix language of the SL defined by the SE. The DFA is 
shown in Figure 1. 

bt 



I 



To ensure that the state update can only be performed exclusively by one 
process, we build a macro called 

wait«and_set (int state; int ...; int in->3n) • 

for n > 0. The macro waits until the condition “state == for some t, 
1 < t < n, becomes true and then sets “state = jt\ Instead of using a 
variable-length list int «i, ; . . . ; int in^jm the actual implementation of 

wait^and_set may use a pointer to a table (an array) for the transitions. 
Here we use the list in order to make the discussion easier to follow. The 
macro is an indivisible (exclusive) operation and can be implemented using 
primitive synchronization mechanisms of the system. 

Then the state transitions of the DFA correspond to the following Sequent 
C statements or macros: 

• Initialization 
-State = 1; 

• The as transition 
wait^auidjset (-State ; 1, 2); 

• The at transition 
wait_jand_set (-State ; 2, 3); 

• The ba transition 
wait.^d-set (-State; 3, 4); 

• The bt transition 
wait-xuid-set (-State; 4, 1); 

The following is the code generated from the ParC program: 

#include<parallel/microtask . h> 
#include<parallel/parallel .h> 

#include<sys/wait . h> 




Fig. 1. DFA for the SE (a->b)* 
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#include<sys/types . h> 

#include<stdio . h> 

main ( ) { 
shared int w; 
int k, r; 

shared int _state=l; 
int f 0 ; 

{ 

int _pw, _pid; 

if ((_pid=forkO) == 0){ 

/* */ 

wait _and_set(_st ate; 3, 4); 
r = w; 

wait.and.set (_state; 4, 1) ; 

/* */ 

exit (0) ; 

} 

else if (_pid < 0){ 

printfC'fork error\n"); 
exit (1) ; 

} 

/* */ 

wait _and_set (.state; 1, 2); 
w = f (k) ; 

wait .cind.set (.state; 2, 3); 

/* */ 

} 

} 



For a slightly more complicated SE “ (a | | b) - >c” , a corresponding DFA 
is shown in Figure 2. 

Then the code generated for the a statement would be 

wait. and.set (.state; 1, 2; 3, 4; 6, 8); 

/* the a statement itself */ 

wait. and.set (.state; 2, 5; 4, 7; 8, 9); 

Similarly, the code generated for b statement would be 

wait. and.set (.state; 1, 3; 2, 4; 5, 7); 

/* the b statement itself */ 

wait. and.set (.state; 3, 6; 4, 8; 7, 9); 
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And the following is the code for c: 

wait_and_set (.state; 9, 10); 

/* the c statement itself */ 

wait .Eind.set (.state; 10, 11); 

With the above two simple examples, the basic idea of implementing SEs 
using SLs should be clear. There are many other issues concerning implemen- 
tation. We mention several of them briefly in the following. 

The first is the size- explosion problem of DFA. For many practical syn- 
chronization problems, the sizes of the DFA can be too large to implement. 
We suggest the use of alternating finite automata (AFA) [2,12] instead of 
DFA in the implementation of SEs. The use of AFA can significantly increase 
the space efficiency. 

The checking of states may become the bottleneck of synchronization since 
it is done sequentially. This is not necessarily true. Note that SEs within 
different concurrent blocks, respectively, can be implemented by automata 
that actually run concurrently. 

Conflicting synchronization constraints and deadlock conditions caused 
by the definitions of SEs in the same scope can be examined by checking 
the intersection of the SLs defined by the SEs and the execution sequences 
defined by the program flow. The precise semantic definition of SEs makes 
such checking conceptually clear and practically possible to implement. 
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Summary. Uniformizing a relation belonging to some family, consists of finding a 
function whose graph belongs to that family and whose domain coincides with that 
of the given relation. Here we focus on relations on finite or infinite strings that can 
be recognized by some version of finite automata. 

Eilenberg showed that rational relations of finite strings have the uniformiza- 
tion property. We examine how this result can or cannot be extended. We show 
that given a length preserving n-ary rational relation and one of its component 
it can be uniformized, when viewed as a binary relation froma this component to 
the remaining components, and that binary deterministic relations also have the 
uniformization property. Finally, we shall show that rational relations on infinite 
strings can be uniformized. 



1 Introduction 

Uniformizing a relation belonging to some family, consists of finding a func- 
tion whose graph belongs to the family and whose domain coincides with that 
of the given relation. Here we are particularly concerned with the relations 
on finite or infinite strings that can be recognized in the traditional sense by 
some type of finite automaton. 

Eilenberg proved in 1974 a uniformization result for rational relations 
on finite strings. Siefkes established in 1975 that the synchronous relations 
on infinite strings enjoy the uniformization property as well. Actually these 
results can be refined to subfamilies of rational relations on finite or infinite 
strings or a mixture of those: to name but the two most important, say the 
deterministic and the synchronous relations. Our purpose is to give a survey 
of all the known results of this type and to show how far they can or cannot 
be extended. In order to more accurately evaluate how lucky we are with 
dealing with strings, suffice it to say that uniformization results on trees no 
longer hold, see [14]. Theoretical computer science and logic have studied the 
subject with different tools. We think it is time to present these results in a 
unifying framework by bringing the two approaches together. We hope the 
reader will be convinced that using both the methods of theoretical computer 
science and those of logic helps greatly simplifying and clarifying some proofs. 
A good illustration is the investigation of the synchronous relations, whether 
on finite or infinite strings, where the language of logic spares some tedious 
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(but of course equivalent) set constructions. This is no wonder since it allows 
us to use “for free” the powerful theory developed by Biichi. 

Historically, the uniformization result on rational relations on finite strings 
can be traced back to [7], i. e., over 30 years ago. It was not stated as such, 
rather it was given a more precise form (technically the function by which 
one can uniformize a given relation is obtained as a composition of a left 
followed by a right “sequential” function). Since then it has been reproved 
with different methods, [1], [16], see also [15] for an account on the subject. 
Eilenberg proved it as a corollary of his “cross-section” theorem, stating in- 
tuitively that it is possible to “rationally” select a representative for each 
equivalence class that intersects a rational subset. This result carries over to 
infinite strings as well. 

As previously said, most of the material here can be considered as “folk- 
lore” by some (actually non so many) researchers. There is one exception 
however: the proof of the uniformization property for rational relations on 
infinite strings seems to be new, [13]. Using a different approach, a similar 
result was independently obtained by D. Beauquier, J. Devolder, M. Latteux 
and E. Timmerman, but has not been published. 

2 Preliminaries 

2.1 Basics on uniformization 

Let us recall a few elementary definitions in order to fix the notations. Given 
two sets X and Y and a (partially defined) function f : X the graph of 
/ is the subset jj/ = {{u,v) | v = f{u)}. The domain dom(R) of a relation 
C X X F is the set of elements x e X for which there exists an element 
y £ Y with (a:, y) G R. The composition of relations is the operation that 
associates with the relations R C X xY and S CY x Z the relation 

Ro S = {(ar, z) £ X X Z \ there exists y £Y, with(x, y) £ R and (i/, z) £ S} 

(1) 

We compose the functions from left to right. 

Now we come to the main definition of this work. Given a family T of 
relations in X x F and R £ uniformizing Ru\ T means finding a function 
fn : X -^Y such that 1) dom(/i?) = dom{R) 2) jj(//?) C R and 3) U(/i?) £ T. 
When no particular mention is given, saying that a relation belonging to a 
family T is uniformizable^ implicitely means that it can be uniformized in T. 



2.2 Finite and infinite strings 

Given a finite alphabet A whose elements are symbols or letters^ we denote 
by A* (resp. A“^) the set of finite (resp. infinite) strings over A. As usual we 
denote by A^ — A* [J A^ the set consisting of the finite and infinite strings. 
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The empty string is denoted by e regardless of which alphabet in relates to 
as no confusion usually arises and \u\ denotes the length of a string with the 
convention |u| = oo whenever u £ . 

This paper is concerned with direct products of sets of the form A* and 

. One of the central tools that is used in this theory is that of hierarchical 
ordering. We recall that given a linear ordering < on an alphabet A, we 
extend it to the free monoid A* by posing u <hier v \u\ < |i;| or if |u| = |v| 
and there exist w,ui,v\ £ A* and a,b £ A*, such that u = waui, v = wbvi 
and a <b holds. 

The notion of lexicographical ordering is more general. Consider a col- 
lection of sets Ei^ where i ranges over an initial segment of the integers or 
over IN. Assume there exists a (possibly partial) ordering <i on each set Ei. 
We endow the direct product Hie/ lexicographical ordering <iex 

defined by lliG/?/* there exists i £ I such that Xj — yj for 

all j < i and X{ <i yi. This construction applies in particular to A^. Indeed, 
this set can be viewed as a collection of copies of A indexed by IN. Any linear 
ordering on A extends to a lexicographical ordering on A ^ . 

As usual we will assume the set A^ is endowed with the product topology 
of the discrete topology on A where the family of subsets of the form 
with u £ A* form a basis of the open sets. It is a standard result that given 
an arbitrary lexicographical ordering on A^ every topologically closed subset 
of A^ contains its lexicographically minimal element. 

3 Relations on finite strings 

3.1 Rational relations on finite strings 

Given an arbitrary monoid M, the least family of subsets containing all 
finite sets and closed under set union, concatenation (i. e., X and Y are in 
X then so is {xy \ x£X,y£Y}) and Kleene closure (i. e., if X is in ^ then 
so is {e} U X U . . . U X® . . .) is the family of rational subsets and is denoted 
by Rat(M). As a particular case, given two monoids M and N, a function of 
M into N is rational if its graph is a rational subset of the product monoid 
M X N. We refer the reader to the two handbooks [5] and [2] for basic results 
in this theory. 

It is well-known that in the case of a direct product of free monoids 
A* X ... X A* , the family of rational subsets, also called rational relations, 
is precisely the family of relations recognized by finite automata. Indeed, the 
notion of finite automaton designed to recognize single strings, was extended 
in the late fifties in such a way as to operate on n-tuples of strings. The idea 
is to provide an automaton with as many tapes as there are components in 
the tuple. 

More precisely assume without loss of generality that the n alphabets Aj 
are disjoint and set A = Ui<t<n^«- We denote by TTi the projection of A* 




62 



C. ChofFrut and S. GrigoriefF 



onto For alH = 1, . . . ,n and by tt the projection onto the direct product 
Al X . . . X A*: 7t{w) = {7Ti{w), . . . jTTniw)). A finite n-tape automaton (we 
shall say also more simply an automaton) is a quadruple A = {Q,I,F,T) 
where Q is the finite set oF states, I C Q is the set oF initial states, F C Q 
is the set oF final states and T C Q x A x Q the set oF transitions. The 
subset oF A| X . . . X A* recognized by Aconsists oF those n-tuples oF strings 
. . . ,7rn(w)) where w is the label oF a successful path, i. e., a path 
starting in an initial state and ending in a final state (see [2, section III. 6] For 
background on n-tape automata where they are called finite transducers). 

The Following is well-known, see [5, Thm IX. 2. 2.] or [2, Thm 4.1]. 

Theorem 1. A relation C A* x . . . x A* is rational if and only if there 
exist a rational subset K C A* such that 

R = {7t{w) I w E K} 

There exists a deterministic version oF such automata but contrarily to 
the free monoids, they are expressively less powerFul than the non determin- 
istic ones. Intuitively, there exists a decomposition Q = Ui<i<n where 
Qi corresponds to the subalphabet Aj. In a state q E Qi, only transitions oF 
letters oF the subalphabet A* are allowed and moreover a given letter defines 
at most one transition. Furthermore the ability to recognize the end oF a 
component is required. More Formally, we assume the alphabets A^ contain 
an extra symbol (ti (the “end-marker” oF the i-th tape). We modify the tt^’s 
by considering them as mappings oF A* into (Aj — ‘^i)* satisfying 7Ti(a) = a iF 
a E Aj — Jtj and 7Ti(a) = e otherwise. 

An automaton is deterministic whenever the transitions satisfy the three 
conditions 



For all {q, a,p), {q, b,r) E T i^ a E Ai and b E Aj then j = i^ 

For all {q, a,p), {q, b,r) E T i^ a = b then p = r ^ (2) 

For all (g^, tii,r) eT if w is the label oF a path leaving r ^ ^ 

then w E (A — A{)* 

A relation is deterministic rational iF there exists a deterministic automa- 
ton in the above sense that recognizes it. Then the Following is a paraphrase 
oF the definition. 

Proposition 1. A relation i? C A* x . . . x A* is recognized by a deterministic 
n-tape automaton if and only if the rational subset K C A* of Theorem 1 can 
be assumed to satisfy the two conditions 

For all u,v,w E A*, iF uv,uw E K,v E A{A*,w E AjA* then i = j 
For all u E A*, iF u^iV E K then v E {A — Ai)* 
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3.2 Synchronous relations on finite strings 

Synchronous relations form an important subfamily the rational relations 
which enjoys nice closure properties. In particular it forms a Boolean algebra 
and some of its properties are decidable, whereas almost all properties of the 
general rational relations are undecidable (Post Correspondence Problem can 
be interpreted as a question on two rational relations). 

Consider a fresh symbol Jt not belonging to the A^’s. With each n-tuple 
(ui,...,Un) € n associate the n-tuple of strings of the same length 

l<i<n 

defined as 

(ui,. with e - max|ui| (4) 

t 

Extending the notation to subsets R C Al x . . . x in the natural way, 
we identify with a subset of strings over the alphabet U = JJ {Ai U 

l<i<n 

{((}). Then the relation R is synchronous if the subset R^ is recognized by 
a Wte automaton over the alphabet Z*. It is not difficult to verify that the 
synchronous relations form a subfamily of the rational relations that is closed 
under the Boolean operations, composition of relations, direct products and 
projections (e. g., [7] where these relations were called FAD-relations or [8]). 
Finally a function f \ A\ x ... x A*^ ^ Bl x ... x is synchronous if its 
graph ti/ is a synchronous relation of A{ x ... x A’^ x B^ x ... x 5^. 

The set of synchronous relations has been logically characterized in [6]. 
For the reader’s convenience we recall the logical language that defines it. The 
signature contains two symbols < and E of binary predicates and a symbol 
Ta of unary predicate for each letter a E A = Aj. The first order 

l<i<n 

language in question is defined on this signature. The individual variables 
belong to the disjoint union of denumerable sets for 1 < i < n. All 
formulae are interpreted as follows. The universe is the union of the A|’s, for 
1 < i <71, and an individual variable x E Xi is interpreted as a string in 
Now < V is true if and only if u and v belong to the same free monoid 
A^ for some 1 < i < n and w is a prefix of v. Furthermore, uEv is true 
if and only if u and v have the same length and finally Ta{u) for some a E A 
is true if and only if the last letter of u is a. To each formula . . . , Xn) 
with set of free variables x\ E Xk^ , . . . , Xn € Xk^ is assigned the set R of all 
n-tuples (ui , . . . , Un) E x . . . x such that (f) is true when each U{ is 
substituted for xi in (f>. It is said that (j) defines R or that R satisfies (j). 

Theorem 2. A subset i? C A* x . . . A* is synchronous if and only if it is 
defined by some formula (f) of the above language. 

As an immediate result we get 
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Corollary 1. Let 0 < k < n be some integer. Each synchronous relation 
R C A\ X ... X can be uniformized by some synchronous function f : 
Alx ...xAl-^ Al+i X ... X 

Proof. Observe first that the hierarchical ordering on the free monoid can be 
easily expressed in the logic. Also, we can express the fact that a n-tuple of 
strings xi , . . . , Xn is lexicographically less than or equal to another 2 / 1 , . . . , i/n • 
Now, let 4>{xi,. . . ,Xn) be a formula defining R. It suffices to associate with 
each fc-tuple of x ... x belonging to the domain of R, the (in the 

lexicographical ordering) least (n - A:)-tuple of Al_^^ x . . . x A* which is 
associated to it. We leave it to the reader to work out the details. 

3.3 Uniformization on finite strings 

Intuitively, Eilenberg’s cross-section theorem, [5, Thm. IX, 7. 1.] asserts that 
given an morphism f : A* B* and a rational subset AT C A*, it is possible 
to “rationally” select a representative among all the elements of K that map 
onto the same element of B*. 

Theorem 3. Let f : A* B'*^ be a morphism and let K be a rational 
subset of A* . Then there exists a rational subset L C K such that f maps L 
bijectively onto Kf. 

This result and its appoach have been widely commented, used and re- 
proven. Traditionally, it has two major consequences: 1) each rational func- 
tion / of a free monoid into another can be recognized by some “unambigu- 
ous” 2-tape automaton (each pair of strings (u,v) with v = f{u) defines at 
most one successful path in the automaton) and 2) all rational relations of a 
free monoid into another are uniformizable which is precisely the result that 
this paper wants to extend to infinite strings, [5, Prop. IX, 8. 2]. 

Proposition 2. Each rational (resp. deterministic rational) relation can be 
uniformized 

Proof. For rational relations this follows from Theorem 1 where the subset 
L of the previous theorem is substituted for K. For deterministic rational 
relations it suffices to observe that condition (2) still holds for all subsets of 
K. 



Observe that the previous result cannot be extended to two or more com- 
ponents. Indeed, consider the following rational relation on the direct product 
{a, by X {a}* X {a}* 

a^, a) I n, m > 0} U {{a^b^, a^,e)\n,m>0} 

and assume there exists a rational function / : {a, 6}* x {a}* {a}* that 

uniformizes it. Let Xq,Xi C {a, 6}* x {a}* be the pre-images of a and e 
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respectively and let Ao and Ai be finite automata recognizing Xq and Xi . 
Denote by m the maximal number of states in these automata and by /x > m 
an integer which is a multiple of the number of occurrences of a (resp. b) in 
any simple cycle of Ao and Ai (a simple cycle is a path where initial and 
final states coincide and where no other state is visited more than once) . Set 

po = max{n € IN | a”)/ = a} and pi = max{n £ IN [ a^)/ = e} 

Assume first po < oo and let N > max{m,po}- Consider € Xq. 

By the pigeon-hole principle applied to the m first states visited by a path 
labelled by there exists a cycle labelled by a pair (a^,a^) with 

p <m. Since /x is a multiple of p, the pair belongs to Xq, 

a contradiction. So we must assume that po = oo. A similar argument shows 
that Pi = oo. For some integer M greater than m H-/x we have (a^b^ ,a^) £ 
Xi. Then {a^~^b^ £ Xq. By the same pigeon-hole principle applied 
again to the m first states visited by a path labelled by {a^~^b^ 
there exists a cycle labelled by a pair (a^, oP) with p <m. Thus (a^b^ ,a^) 
belongs to Xq, a contradiction. 

Using similar techniques we would prove that XUY C {a}* x {a}* x {a, b}* 
with 

X = 1 m,p > 0} and Y = {(a^^P,a^,b^aP) 1 m,p > 0} 

is a rational relation which cannot be uniformized by any rational function 
of {a}*" X {a}* into {a, 6}*. However, when all alphabets are unary, ratio- 
nal relations can be uniformized for any subset of components. This follows 
trivially from the fact that such rational relations are defined by the logic of 
Presburger arithmetic, [9]. 

Proposition 3. Let 1 < k <n be some integer and let Ai be unary alphabets 
for z = 1, . . . ,n. Each rational relation RC A\x . ..x can be uniformized 
by some rational function f : Al x ... x A^ Al_^^ x ... x A’^. 

4 Relations on infinite strings 

Biichi generalized in [3] the notion of finite automaton in order to have it 
operate on infinite strings. The family of subsets of recognized by some 
Biichi automaton in this manner is denoted by Rat A^ and is called the 
family of rational subsets of infinite strings (this is justified by the fact that 
this family is closed under extended “rational” operations, [5, Thm. XIV. 4. 
1.]). In the same way traditional finite automata can be used to recognize 
relations on finite strings, Biichi automata can be used to recognize relations 
on infinite strings. We refer the interested reader to [11] for a thorough study 
of these relations. Here, we will only recall what is necessary for our purpose. 
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4.1 Rational relations on infinite strings 

A finite n-tape automaton A = (Q, /, F, T) can be transFormed into a Biichi 
automaton and used to recognize n-tuples oF possibly infinite strings by in- 
terpreting F as a set oF repeated states. The definitions oF paragraph 3.1 carry 
over here naturally. An infinite path is successful iF it starts in an initial state 
and visits infinitely often a repeated state. The subset oF x . . . x rec- 
ognized by the automaton is the set oF n-tuples 7t{w) = (7Ti(n;), . . . ,7Tn{w)) 
where w £ is the label of a successful path in the automaton (for all 
z = 1, . . . ,n the projections iTi extend trivially from A^ to A^). 

As seen in section 3.3, the key argument for Eilenberg’s uniformization 
result is the cross-section theorem along with what he calls the first factor- 
ization theorem (Theorem 1). For infinite strings we obtain the same result 
via an extension of his “second factorization theorem” which shows that all 
rational relations is the somposition of a synchronous relation followed by 
some rational substitution [5, Thm IX. 5.1.]. 

We recall that a morphism of F* into A* is alphabetic if it associates with 
every letter of F a letter of A. A substitution of B* into a monoid M is a 
morphism of into the power set of all subsets of M. In the case where M 
is a direct product of free monoids A^ x . . . x A* , the substitution can be 
extended from B^ to A^ x . . . x A^. Hereafter we deal with substitutions 
into Ai X ... X A* that are rational, i. e., that map B* into Rat(A* x . . . x A* ) 

Theorem 4. [10, Prop. 2. 1.] Given a relation R C Af^ x ... x A^ the 
following conditions are equivalent. 

1) R is recognized by some Biichi automaton. 

2) there exist a finite alphabet B, a rational subset K C B'^ , an alphabetic 
morphism cp : B* A^ and a rational substitution ip : B* Rat(A .2 x . . . x 
A* ) such that R = o C\K o 'll) holds, where C\K is the retriction of the 
identity to the subset K. 

The subsets of Af^ x . . . x A^ recognized in this manner are called rational 
relations. It can be readily verified that for all rational relations R C Af^ x 
... X A^ the relation RC\ A^ x ... x A^ is also rational. More generally, the 
following holds (e. g., [11]). 

Proposition 4. For i = l,...,n let Si = A^ or Si — Af . Let R be the 
relation recognized by a Biichi automaton A. Then ROSi x. . .xSn is rational. 

From now on we deal with “purely infinite relations” only, i. e., with relations 
in Ay X ... X AJ:^. As for finite strings, the notion of deterministic automaton 
exists. A deterministic automaton satisfies the following conditions 



1) for all {q, a,p), {q, b,r) eT if a e Ai and b € Aj then j = i 

2) for all {q, a,p), (q, 6 , r) € T if a = 6 then p = r 
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A relation R C x ... x is deterministic if there exists a determin- 
istic automaton that recognizes it. It is synchronous (resp. deterministic 
synchronous) if viewed as a subset of {A\ x x it is recogniz- 

able by some Biichi (resp. deterministic Biichi) automaton on the alphabet 

Ai X . . . X An. 



4.2 Uniformization on infinite strings 

The main result of this paper (Theorem 5) is based on the following property 
which shows that synchronous relations can be uniformized. 

Proposition 5. Let 0 < k < n he some integer. Each synchronous relation 
R C A^ X ... X A^ can be uniformized by some synchronous function f : 

AuJ w w Au) V A(jJ w w AU 

^1 ^ ^ -r A ... A . 

Proof. We first verify that it suffices to consider the case n = 2, A: = 1. Indeed, 
consider two bijections a : A\ x . . . x Ak A and (3 : Ak+i x ... x An B 

where A and B are new subsets. By identifying A^ x .. .xA'^ with (Ai x . . . x 

j we may extend a to an isomorphism of Af x. . .xA‘^ onto A"^ . Similarly 
we extend /3 to an isomorphism of x . . . x onto B^. The relation 
a~^oRo/3 C X is synchronous and can be uniformized by some function 
/ : A^ Then the function aofof3~^ : A^ x . . .x A^ x • • • x 

uniformizes R as it can be readily verified. 

Prom now on we deal with a synchronous relation R C A^ x B^ recognized 
by some Biichi automaton A = (Q,I,F,T). It is convenient, given a pair 
(u,v) G A^^ X to say u is the input and v is the output component. A 
run is a finite or infinite sequence of states {qi)i<n, n < oo, visited in a 
path of the automaton, i. e., for which there exist (ui^Vi) G A x B such that 
(qi,Ui,Vi,qi^i) G T holds for all i < n. Without loss of generality we may 
enforce the following additional condition which guarantees that the output 
is uniquely defined by an input and a run 

for all O', p G Q, a G A and , 62 € -B 
if (g,a,6i,p),(gf,a,62,p) ^ T then bi = 62 

Our proof follows the usual pattern. It consists of selecting for each input 
string u G A^ a, specific image v G B^ satifying {u.,v) G R in such a way 
that the selection can be performed by a finite automaton. The initial idea 
of Eilenberg of choosing v as the minimal string in some prescribed lexico- 
graphical ordering does not carry over to infinite strings since whatever the 
ordering chosen, there might not exist a minimal element associated with an 
input (e. g., the relation consisting of the pairs (a^^a'^b^) and (6^,6^a^) for 
all n > 0). 

In the present situation we show that to any arbitrary string u G A^ in the 
domain of the relation, we can assign a unique string v G B^ with (u,v) G R 
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through a second order monadic Formula. However, contrarily to Eilenberg’s 
approach, instead oF selecting the image through some oF its properties we 
choose it via a run oF the automaton. Among the runs determined by the 
input string, we choose that which visits repeated states earliest (hence the 
term ‘‘greedy ordering” see below), and whenever this does not suffice to 
single out one run, we will choose the minimal in the lexicographical ordering. 
Thus, iF (j)(u,v) is a monadic second order Formula defining the relation R (i. 
e., R = {{u,v) ^ X | (t>{u,v) = true}), then the uniFormization is 
expressed by the Following monadic second order Formula 

For all u £ V G the three conditions hold 

1) (f){u, v) is true 

2) there exists a run ^ with label (w, v) 

3) For all runs t] with label {u, w) For some w inequality ^ <7j holds 

More precisely, we consider a linear ordering < on Q under which the set 
F is an initial segment {q £ F and p < q implies p £ F) and we denote by 
<iex the lexicographical extension oF < to . Let T be a new symbol and 
consider the ordering on the set F U {T} which is the trace oF < on F and 
For which T is the greatest element. Extend this ordering to a lexicographical 
ordering on the infinite sequences on the alphabet F U {T} and denote this 
new ordering by <f- Let finally ttf : (F U {T})^^ be the substitution 

defined by: 

n-TT - / ^ if g e F 
^ ^ ^ T otherwise 

The greedy ordering <greedy on (Q*F)^ is defined by setting rj <greedy ^ 
iF and only iF 

<F or ( T]7TF = ^ff and t] <iex ^ ) (7) 

We leave it to the reader to verify that <greedy is indeed an ordering. 

Consider an input u and let Acceptt^ bo the set oF successFul runs asso- 
ciated with it. We assume Accept ^ is non empty and we shall “construct” 
its < greedy -minimal element. We start with defining the ordering ^ on the 
set (Q — F)*F by posing x < y \i one oF the Following conditions holds: 1) 
|x| < \y\ or 2) |x| = \y\ and their last occurrences £ F satisFy x' < y' or 
3) \x\ = \y\ and x' = y‘ and x <iex 2/- 

Let x\ £ {Q — F)*F be the -<-smallest element in (Q — F)*F which is 
the prefix oF some run in Accept^ and let Si C Accept^ be the non empty 
set oF successFul runs starting with xi. Now let X 2 € (Q — F)*F be the 
smallest string such that X\X 2 is the prefix oF some successful run in Si and 
let S 2 C Si be the non empty set oF successFul runs starting with xiX 2 - The 
process continues and defines an infinite string X 1 X 2 . which is the <greedy- 
minimal element oF Accept^. 

Because oF condition (6) we have defined in this way a mapping f : A^ 

B^ by setting f{u) = v where v is the output associated with the run p. 
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Since the greedy and the lexicographical orderings are monadic second order 
definable, so is the function / which completes the proof. 

We are now ready to prove the result for arbitrary rational relations. 

Theorem 5. Let 0 < i < n. Every rational relation on x . . . x can 
he uniformized by some rational function : f : Af 

Proof Indeed, by Theorem 4 every relation R can be factorized into R = 
o r\K o %!) where is a finite alphabet, K £ Rat.B’", (p : B* is an 

alphabetic morphism and : J5* -> Rat(A 2 x . . . x A* ) a rational substitution. 
Choose for all 6 € B an arbitrary element in brp and let -> A 2 x . . . x A* 

be the resulting morphism. It suffices to show that o r\K o can be 
uniformized. However the relation o OK is synchronous. By the previous 
theorem, it can be uniformized by a function / : A^ The function 

/i? = / o uniformizes the relation R. 

The same counter-examples of paragraph 3.3 can be adapted to infinite 
strings (by completing every finite string with a special symbol infinitely 
repeated) to show that uniformization in more than one component does not 
hold in general. Another interesting consequence is the fact that the cross- 
section property holds for infinite strings. 

Corollary 2. Let f : B^ be a morphism and let K be a rational 

subset of A ^ . Then there exists a rational subset L C K such that f maps L 
hijectively onto Kf. 

Proof Indeed, the relation C\Kf o f~^ o fiRT C x A^ is rational. There 
exists a rational function g : B^ A^ that uniformizes it, i. e., that selects 
for each element x £ Kf a, unique element in y £ K with x — yf. Then 
L = Kfg £ RatA^ and / maps hijectively L onto Kf. 

Also, since all rational subsets of infinite strings are unambiguous, [4], we 
have 

Corollary 3. Every rational function f : A^ B^ x . . . x B^^ is unambigu- 
ous. 

We observe, as a easy negative result, that the family of topologically 
closed rational relations cannot be uniformized. Indeed, consider the closed 
subset on the alphabets A = JB = {a, 6} 

(a^ X A^) U {(a^6s,6s) | n > 0,5 E A^} 

If there would exist a function that would uniformize the relation, it would 
be continuous, but this is clearly impossible. 
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4.3 A topological interpretation 

The proof of uniformization for synchronous relations can also be seen as a 
proof of the following result. 

Proposition 6. On every rational rational subset X of there exists a ra- 
tional linear ordering < such that every non empty Y C X which is relatively 
closed in X has a smallest element. 

Proof. Let A = {Q, I, F,T) be a Biichi automaton recognizing X C . We 
assume without loss of generality that the existence of two transitions of the 
form (q,a,p), {q, b,p) € T implies a — b.k run is a finite or infinite sequence 
of states {qi)icnj n <oo, visited in by path of the automaton, i. e., for which 
there exist ui £ A such that {qi^Ui.,qiJ^i) £ T holds for all i < n. Because 
of the previous condition, there exists at most one string in associated 
with a given run. A run is successful if it visits infinitely often some repeated 
state. For every non empty subset Y C X we denote by Accepty the set of 
all successful runs associated with some element in Y. 

As in Theorem 5 we can construct an infinite string • • • € which 
is the <greedy-niinimal element of Accepty. This string is a successful run on 
some input u £ X. Assume Y is of the form Y = X D Z for some subset 
Z C A^ . Then u belongs to the adherence of Z. If Y is relatively closed, i. 
e., if we may assume furthermore that Z is closed, then u belongs to Y. 

Now define a linear ordering ^ on X as follows : v and only if the 

<greedy-niinimal element of Accept^; is smaller than the < greedy -minimal ele- 
ment of Acceptti;. If y C X is relatively closed in X then the string u £ Y 
obtained as explained above is clearly the ^-smallest element of Y. 

This result can be viewed as the automaton version of a general topological 
result which states that on every Bor el subset X of A^ (in fact, also on every 
analytical subset) there exists a linear ordering such that every non empty 
relatively closed subset Y C X has a smallest element. The proof of this 
result also uses an auxiliary greater topological set. A classical result (see 
[12, Thm 37. 1.]) states that X is a continuous image of the Baire space 
hence the projection of some closed R C A^ x The lexicographical 
ordering on A^ x is a linear ordering such that every non empty closed 
set has a smallest element. It induces on X the wanted ordering defined as 
follows : X ^ 2/ iff smallest element of i? D ({x} x is smaller than the 
smallest element of D {{y} x o;^). 

Observe that the Baire space cannot be replaced by any space 
with B finite since continuous images of compact spaces are compact. 

In fact, in the proof of the above Proposition, the Baire space does occur 
implicitely in the definition of the greedy ordering. This can be seen as follows. 
There is a natural injection fi:om the space of successful runs into the product 
X X which maps a successful run onto the triple consisting of the 
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sequence of positions of states in F, the sequence of successive states in F 
and the sequence of successive states. The greedy ordering on successful runs 
then corresponds to the lexicographic product of lexicographic orderings on 
the components. 
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Summary. The tree languages accepted by (finite state) tree-walking automata 
are known to form a subclass of the regular tree languages which is not known to be 
proper. They include all locally first-order definable tree languages. We allow the 
tree- walking automaton to use a finite number of pebbles, which have to be dropped 
and lifted in a nested fashion. The class of tree languages accepted by these tree- 
walking pebble automata contains all first-order definable tree languages and is 
still included in the class of regular tree languages. It also contains all deterministic 
top-down recognizable tree languages. 



1 Introduction 

One of the questions in tree language theory is whether a natural sequen- 
tial automaton model exists for recognizing the regular tree languages. Of 
course, by definition, they are recognized by the bottom-up finite tree au- 
tomaton. But that automaton is essentially parallel rather than sequential: 
the control of the automaton is at several nodes of the input tree simul- 
taneously, rather than at just one. The top-down finite tree automaton is 
also parallel and, moreover, its deterministic version does not recognize all 
regular tree languages, which seems unnatural when compared to the case 
of strings. For strings, the (1-way, on-line) finite automaton was generalized 
to the 2- way (off-line) finite automaton, which also recognizes exactly the 
regular languages, see [16,15]. Here the point of view has changed: the in- 
put is not fed into the automaton (like money into a coffee machine), but 
the automaton walks on the input string (like a mouse in a maze). Clearly, 
this can be generalized to a sequential finite automaton on trees: the tree- 
walking automaton, introduced in [1]. The finite control of the tree- walking 
automaton is always at one node of the input tree. Based on the label of that 
node and on its child number (which is i if it is the i-th child of its parent, 
with z = 0 for the root), the automaton changes state and steps to one of 
the neighbouring nodes (parent or child). The tree- walking automaton of [1] 
was equipped with an underlying context-free grammar and with an output 
tape, to model syntax-directed translation from strings to strings. It did not 
need the test on the child number because that information could be coded 
into the nonterminals of the context-free grammar. As shown in [14], without 
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the child number test the tree-walking automaton is not able to search the 
input tree in a systematic way, such as by a pre-order traversal. For this and 
other reasons the child number test is an essential feature of the tree- walking 
automaton. 

Unfortunately, it is an open problem whether the tree- walking automa- 
ton recognizes all regular tree languages, and we conjecture that it does not, 
cf. [9]. Assuming the conjecture, there are two natural questions. How does 
the class TWA of tree languages accepted by tree-walking automata compare 
to other well-known subclasses of the regular tree languages? Which natural 
features can be added to the tree-walking automaton to obtain an automa- 
ton that does recognize the regular tree languages? To start with the second 
question, the main trouble with the tree- walking automaton seems to be that 
it gets lost rather easily: in a binary tree of which all internal nodes have the 
same label, all nodes look pretty much the same. One way of solving this is to 
extend the tree- walking automaton with a synchronized pushdown, for which 
pushing and popping is synchronized with moving down and up in the tree, 
respectively. It is shown (implicitly) in [10] and (explicitly) in [14] that this 
automaton recognizes exactly the regular tree languages. Another, classical 
remedy against getting lost is to use pebbles. For instance, arbitrary mazes 
can be searched by “maze- walking” finite automata with two pebbles, see 
[5]. The main aim of this paper is to investigate the power of tree- walking 
automata with pebbles. Obviously, the unrestricted use of pebbles leads to a 
clciss of tree languages much larger than the regular tree languages, in fact 
to all tree languages in NSPACE(logn). Thus, we restrict the automaton 
to the recursive use of pebbles, in the sense that the life times of pebbles, 
i.e., the times between dropping a pebble and lifting it again, are properly 
nested. A similar, but stronger, nesting requirement is studied in [13] for 
2- way automata on strings. We prove in Section 5 that our restriction indeed 
guarantees that all tree languages recognized by the tree-walking pebble au- 
tomaton are regular, but we conjecture that the automaton is not powerful 
enough to recognize all regular tree languages. In Section 6 we generalize the 
notion of pebble to that of a “set-pebble” , in such a way that the tree- walking 
set-pebble automaton recognizes exactly the regular tree languages. 

To answer the first question, we compare the TWA languages with the tree 
languages that can be defined in first-order logic, see [19] for a survey. One 
of the reasons that the regular tree languages are the natural generalization 
of the regular string languages to trees, is that they are precisely the tree 
languages definable in monadic second-order logic. This was, in fact, the 
main motivation for the introduction of finite tree automata in [7,18]. Thus, 
one way of investigating the power of several types of tree- walking automata 
is to compare them to several types of logics on trees. We show in Section 3 
that TWA contains all tree languages that are definable in locally first-order 
logic, where ‘locally’ means that the logic can talk about the parent-child 
relation between nodes of the tree, but not about the ancestor relation. We 
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conjecture that it does not contain all first-order definable tree languages, 
but show in Section 5 that they can be recognized by the tree- walking pebble 
automaton. Note that TWA contains tree languages that are not first-order 
definable, such as the set of all trees with an even number of nodes (cf. [19]). 
As another answer to the first question, we conjecture in Section 4 that TWA 
does not contain all deterministic top-down recognizable tree languages, but 
show that they can be recognized by the tree- walking pebble automaton, with 
one pebble only. 



2 Preliminaries 

In this section we recall some well-known concepts and results concerning 
trees, logic for trees, and tree- walking automata. We use N = {0, 1,2, . . .}, 
and for m, n G N, [m, n] = {i\m <i <n). 

Trees. For a ranked alphabet i7, i.e., an alphabet U together with a 
function rank : U N, the set of all trees over U is denoted Tj;. A subset of 
Tx! is called a tree language. As usual, a tree t over U is viewed as a finite, 
directed graph of which the nodes are labelled with symbols from U. By Vt 
we denote the set of nodes of t. Each node v £ Vt has k children where k 
is the rank of the label of v. There is a linear order on these children which 
allows us to speak about the i-th child of v, and there is an edge with label 
i from V to its i-th child (for 1 < i < k). The child number of a node i; is i if 
it is the i-th child of its parent, and 0 if is the root of t. For nodes u and 
V of t, u < V denotes that u is an ancestor of v, i.e., that there is a directed 
(possibly empty) path from u to v. The yield of t is the string obtained by 
concatenating the labels of its leaves, from left to right. Finally, trees are 
denoted by terms in the usual way: (r{ti, ... ,tk) is the tree of which the root 
has label a (of rank k) , with direct subtrees , . . . , . 

Logic for Trees. We consider the same types of logic as in [19], viz. 
monadic second-order (mso) logic, first-order (fo) logic, and locally first- 
order (lfo) logic. In [19] the latter two are called FO[<] logic and FO[S] 
logic, respectively. 

For a ranked alphabet S, we consider monadic second-order formulas 
over U that describe properties of trees over U. This logical language has 
node variables x,y ,.. and node-set variables X, F, . . .. For a given tree t 
over X, node variables range over the elements of Vi, and node-set variables 
range over the subsets of Vt- There are five types of atomic formulas over E: 
\ah(r{x), for every cr £ E, meaning that x has label cr; edgj(x,2/), for every 
i < the rank of a symbol in E, meaning that the i-th child of x is y; x < y, 
meaning that x is an ancestor of y\ x — y^ and x G X, with obvious meaning. 
The formulas are built from the atomic formulas using the connectives -i, 
A, V, and as usual. Both node variables and node-set variables can 
be quantified with 3 and V. We will use edg{x,y) for the disjunction of all 
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edg,(x, y) (meaning that x is the parent of y), root(ar) for -i3y{edg{y, x)), and 
leaf(a:) for -^3y{edg{x,y)). 

A first-order formula is an MSO formula that does not contain node-set 
variables, and a locally first-order formula is an fo formula that does not 
contain atomic formulas x <y. 

For a closed formula <j) over U and a tree t e Tjj, t \= (p denotes that 
t satisfies (f>. If a formula (f> has free variables, say, x,y,X, we also write 
<l>{x,y,X). Moreover, t |= (f>{u,v^U) denotes that t satisfies (p when x,y,X 
have value u, v, U, respectively (with u^v eVt and U CVt). The tree language 
defined by a closed formula (p over X is L{(p) — {t £ Tx; \ t \= (p). L{(p) is called 
an MSO definable tree language. If (p is first-order, or locally first-order, then 
L{<p) is called an fo definable, or lfo definable tree language, respectively. 
According to the classical result of [7,18] (first shown for strings in [6,8]) the 
MSO definable tree languages are precisely the regular tree languages, i.e., the 
tree languages accepted by (bottom-up or top-down) finite tree automata. 
For more information on regular tree languages, see, e.g., [11,12]- 

We will also consider “trips” on trees (cf. [2,3,9]). A trip, or node relation, 
over is a set of triples (t,u, v) where t is a tree over X, and u,v eVt. The 
trip defined by an MSO formula (p{x, y) with two free node variables x and y, 
is T{(p) = {{t,u,v) 1 1 ^ (p{u,v)}. T{(p) is called an MSO definable trip. In [9] 
the MSO definable trips are called the regular trips. 

TVee- Walking Automata. A tree-walking automaton, or tw automaton, 
for short, is a (nondeterministic) finite state device that walks on a tree from 
node to node, following the edges of the tree (in either direction). At each 
moment of time the tw automaton is in a certain state, at a certain node of 
the input tree (over some ranked alphabet X). In one step, it can test the 
label and the child number of the current node, and move to the parent or to 
one of the children of the node, changing state. A child can be specified by 
its child number. The language L{A) accepted by a tree-walking automaton 
A consists of all trees (over X) on which A has a computation that starts at 
the root of the input tree in its initial state, and ends in a final state. L{A) 
will be called a twa language. 

Since the tw automaton can test the child number of the current node 
(and hence, in particular, can test whether or not it is at the root), one 
of its basic capabilities is to make a pre-order traversal of the input tree 
(deterministically), starting and ending at the root: when entering a node v 
for the first time (from above) , it moves up if v is a leaf, and otherwise moves 
down to v^s first child; when entering v fi'om below, it knows the number of 
the child u it just left (by testing u^s child number and storing it in its finite 
control) and thus can move down to v^s next child, or move up if u was the 
last child of v. 

In [2,3] the tree-walking automaton is extended with logical capabilities: 
a tree-walking automaton with MSO tests is a finite state device as described 
above, with the additional possibility of testing MSO properties of the current 
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node. An MSO property is an MSO formula ^{x) with one free node variable; 
for the current node u of the input tree the automaton can test whether 
t 1= 4>{u). A given automaton can of course use only finitely many of these 
MSO tests. The language L{A) accepted by such an automaton A is defined 
as above, but the main concern in [2,3] is with the trip T{A) computed by 
A\ the set of all (t^u^v) such that A can walk from to v on t, starting in 
its initial state and ending in a final state. The main result of [2,3] is the 
following. 

Proposition. A trip is MSO definable if and only if it can be computed by a 
tree-walking automaton with MSO tests. 

It is an immediate consequence of (the if-direction of) this result that 
all tree languages accepted by tree- walking automata with MSO tests are 
MSO definable: if T{A) = T{(f){x,y)), then L{A) = Li'ip) where ^ is the 
formula 3x,y : root(ar) A (j){x,y). In particular, all twa languages are MSO 
definable and hence regular. However, as mentioned in the Introduction, it 
is an open problem whether every regular tree language is a twa language. 
Note that every regular tree language is a projection of a deterministic twa 
language. This is because every regular tree language is a projection of the set 
of derivation trees of a context-fi:ee grammar (cf. [12], Section 8), and every 
such derivation tree language can obviously be accepted by a deterministic 
tw automaton: the automaton traverses the input tree and checks for each 
node whether the labels of the node and its children form a production of 
the context-free grammar. 

3 Locally First-Order Logic 

As mentioned above, it is not known whether the tree-walking automaton can 
accept all mso definable tree languages. Here we show that it can accept all 
locally first-order definable tree languages. The hard work for this has already 
been done: the proof is based on the characterization of the lfo definable 
tree languages as the so-called locally threshold testable tree languages ([19], 
Section 4.2). Intuitively, a tree language is locally threshold testable if mem- 
bership of a tree in the language can be determined by looking at local spheres 
around all nodes of the tree, and count how many times these spheres occur, 
up to some threshold. To explain this formally we need some terminology. 

Let be a ranked alphabet. Consider a “radius” r £K For a node u of a 
tree t over E, we denote by sph,.(t, u) the subgraph of t induced by all nodes of 
distance at most r ton (where distance is measured along undirected paths), 
with n as a designated node. Define Sr = {sphy,(t,it) | t G Tz;,u G Vi}. This 
is a finite set because we do not distinguish between isomorphic graphs. Now 
consider a “threshold” g G N. Define Fr^g to be the (finite) set of all partial 
functions f : Sr ^ [O,^]. For a tree t, define fi g G such that for every 
sphere s G 5^, fl^g{s) = #{u G Vi | sph^(f,t/) = s} provided this number is 
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in [0,^]. Now, a tree language L over E is locally threshold testable if there 
exist r, g E N and F C Fr^q such that L = {t E | f^^q E F}. 

It should be clear that every locally threshold testable tree language can 
be accepted by a deterministic tw automaton. The automaton A traverses 
the input tree t in pre-order and for each node u of t it computes the sphere 
sph^(t,u) with centre u, in its finite control, by just searching systematically 
the neighbourhood of u within distance r. Thus, counting the number of 
occurrences of these spheres up to the threshold q, A can compute ^ and 
check whether it is in F. 

This shows that the deterministic tree-walking automaton can accept all 
LFO definable tree languages. 

4 One Pebble 

We conjecture that the twa languages form a proper subclass of the regular 
tree languages (cf. [9]) and propose the following tree languages and 
as counter-examples. Let E = {(j, a,6} where a has rank 2 and a,b have 
rank 0. Let Li be the set of all trees t G such that all root-to-leaf paths 
of t have even length, and let L 2 be the set of all trees t £ Tjj such that 
the yield of t is in a*ba*. For i = 1,2, the language consists of all trees 
cr{ti,a{t 2 , . . .cr{tk,T ) . . .)) with A: > 0, r E {a,fe}, and ti, . . . ,tk € Li. Thus, a 
tree t in consists of a spine^ i.e., the path from the root of t to its right- 
most leaf, such that the first child of each node on the spine is the root of a 
subtree which belongs to L*. It should be clear that and are regular 
tree languages. We think that they cannot be accepted by a tw automaton. 

Intuitively, the reason that no tree-walking automaton A can accept L®*^ 
is the following. Let us say that a subtree of the input tree t is a spine subtree 
if its root is the first child of a node on the spine of t. Thus, A has to check 
that every spine subtree of t is in L{ . One way of doing this would be to move 
down node by node from the root along the spine, and for each node u on the 
spine check the spine subtree 5 of u. To do this, A has to visit all the leaves 
of s. But then A gets lost because it does not know when it has returned to 
the spine, i.e., to u. The only way for A to find out whether a node v is on 
the spine seems to be to move up as long as the child number is 2 and then 
test whether it is at the root, but then of course A has also got lost because 
it does not know how to return to v. 

Note that it is not diflScult to see that the complement of L®^ can be 
accepted by a tree-walking automaton: walk down nondeterministically to the 
root of one of the spine subtrees and then walk down nondeterministically to 
one of its leaves, checking that the path has odd length. Thus, if L®^ cannot be 
accepted by a tree-walking automaton, then the class of twa languages is not 
closed under complement. Moreover, it would imply that the deterministic tw 
automaton is less powerful than the nondeterministic one, because the class 
of deterministic twa languages is closed under complement. The latter follows 
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from the, maybe surprising, fact that every deterministic tw automaton can 
be simulated by one that is always halting (see [17]). 

As mentioned in the Introduction, one way of not getting lost is to use 
pebbles. It should be clear that can be accepted by a (deterministic) 
tree- walking automaton with one pebble: the pebble is put on a spine node 
while the automaton is checking the corresponding spine subtree. A one- 
pebble tree- walking automaton (shortly, Ip-tw automaton) is a tree- walking 
automaton that additionally carries a pebble. It can drop the pebble on the 
current node, it can test whether or not the pebble is at the current node, 
and it can lift the pebble from the current node (when it lies there, of course). 
At the beginning and end of a computation the pebble should not be on the 
input tree. A Ip-tw automaton A accepts a tree language L{A) in the usual 
way, and L{A) is called a Ip-twa language. It is well known that 2-way finite 
automata (on strings) with one pebble recognize the regular languages, see 
[4,13]. Similarly, it is shown in (Theorem 10 of) [9] that all Ip-twa languages 
are regular tree languages, see also the next section. 

We now note that the language is, in fact, a deterministic top-down 
recognizable (dtr, for short) tree language, i.e., can be recognized by a de- 
terministic top-down finite tree automaton (see, e.g., [11,12]). In fact, we 
will show that every dtr tree language can be accepted by a (deterministic) 
one-pebble tree-walking automaton. This is based on a well-known charac- 
terization of the dtr tree languages (see, e.g., [11], Theorem II.11.6) which 
we now recall. For a ranked alphabet E, we define the (non-ranked) path 
alphabet Us to consist of all symbols ct* with a e E and 1 < i < rank(o-), 
plus all symbols of E of rank 0. Thus, for the ranked alphabet E of Lsp, 
IJx; = {cri,a 2 ,a,b}. For a tree t, its path language path(t) C is defined 
recursively as follows: for r of rank 0, path(r) = {r}, and for a of rank A; > 0, 
path(cr{ti, . . . ,tk)) = U*=i • path(tj). 

The characterization is: a tree language L over E is a. dtr tree language 
if and only if there is a regular (string) language R over II x; such that L = 
{t e Tx \ path(t) C i?}. As an example, the regular language R for Lsp is 
(7-2 (a Ub\J cr I R) where R is the set of all strings of odd length. 

Using this characterization, it is easy to see that all dtr tree languages are 
deterministic Ip-twa languages. Let i? be a regular language over II x and 
let M be an ordinary deterministic finite automaton that accepts the mirror 
image of R. We describe a one-pebble tw automaton A that checks whether 
all paths of the input tree t are in R. The automaton A traverses t in pre- 
order and for each leaf u of t it executes the following subroutine. It drops 
its pebble on u and walks up to the root, simulating M on the corresponding 
string in path(t). More precisely, it starts with the initial state of M and feeds 
M the label of u as input. For each node v on the path from u to the root, 
A determines the child number i of v, moves to the parent v' of u, and feeds 
M the symbol ai as input, where a is the label of v'. At the root, A checks 
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that M has arrived in a final state. Finally, A returns to u by searching t for 
the pebble (traversing t deterministically), and lifts the pebble from u. 

This shows that all deterministic top-down recognizable tree languages 
are accepted by deterministic tree-walking automata with one pebble. 

We conjecture that the Ip-twa languages are still a proper subclass of the 
regular tree languages. The proposed counter-example is the language 
which is obtained from L = by substituting p{L) for each a and p{L^) 
for each 6, where /? is a symbol of rank 1 and is the complement of L. 
Intuitively, to recognize a tree from a Ip-tw automaton would need its 
pebble to recognize the outer occurrence of a tree from L and thus would 
have to behave like a tw automaton to recognize the nested occurrences of 
trees from L and L^. Note that it is easy to accept with a tree- walking 
automaton that uses two pebbles. 



5 Nested Pebbles 

Next we note that the language of Section 4 is first-order definable (but 
not locally). To define it, we use x <2 y to mean that x <y and that all edges 
on the path from x toy have label 2. The latter property is expressible by the 
formula Vx', 2/^ : {x < x' Aedg(x' ,y') Ay^ < 2 /) edg2 (x' , 2/O ■ Now = L( 0 ) 

where 4> is the formula "ix,y,z : (root(ar) Ax <2 y A edgi(y,z)) -> 'ip{z), and 
ij){z) is 3!?/ : z < ?/ A leaf(t^) A laht{u). Similarly, it is easy to show that the 
language discussed at the end of Section 4, is first-order definable too. 

As observed in Section 4, can be accepted by a tw automaton with a 
pebble, and by one that uses two pebbles. Since we know from Section 3 
that tw automata can recognize all locally first-order definable languages, this 
suggests that tw automata with pebbles can recognize all first-order definable 
languages. The key idea here is that a quantified variable corresponds to a 
pebble. However, as mentioned in the Introduction, the unrestricted use of 
pebbles leads out of the class of regular tree languages. Since quantifiers in 
a formula are properly nested, it seems natural to require that the life times 
of the pebbles are nested, where a life time of a pebble is the time between 
dropping it on a node and lifting it again (note that a pebble, like a cat, 
usually has more than one life). The obvious automaton for indeed has 
nested pebble life times. 

A tree-walking automaton with nested pebbles (shortly, p-tw automaton) 
is a tree- walking automaton that carries a finite number of pebbles, each with 
a unique name. In one step, it can determine which pebbles are lying on the 
current node, lift some of them, and drop some others. Initially and finally 
there should be no pebbles on the input tree. Moreover, the life times of the 
pebbles should be nested. This can be formalized by keeping a pushdown of 
pebble names in the configuration of the automaton, pushing a pebble when 
it is dropped and popping it when it is lifted. Note that since each pebble 
name occurs at most once in the pushdown, the automaton may keep track 
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of the pushdown in its finite control and thus “know” when it is allowed to 
drop or lift a pebble. A p-tw automaton A accepts a tree language L{A) as 
usual, and L{A) is called a p-twa language. We will also need the trip T{A) 
computed by A (cf. Section 2): the set of all (t^u,v) such that A can walk 
from u to u on t, starting in its initial state and ending in a final state (with 
the above restrictions on pebbles). 

In the remainder of this section we show that all first-order definable tree 
languages are deterministic p-twa languages, and that all p-twa languages 
are regular. The first proof is by induction on the structure of the formula. 
For each first-order formula 0 we construct a deterministic always-halting 
p-tw automaton A that uses the (free or bound) variables of (j> as pebble 
names. Without loss of generality we may assume that no variable occurs 
both free and bound in (f>. For a given input tree t, the automaton A finds 
out whether or not (j) is true for t. More precisely, let the free variables of 
(j) be in the set {ari, . . . ,Xjfe}, i.e., </>(xi, . . . ,Xjfe). For nodes ui,. .. ,Uk of t, A 
receives as input the tree t with the pebbles xi,. . .^Xk lying on the nodes 
ui, . . . respectively, and starts its computation in its initial state at the 
root of t. During its computation it is not allowed to lift or drop the pebbles 
Xi (but they can of course be tested); the life times of the other pebbles 
should be nested. The computation of A should halt at the root of t with the 
pebbles xi,. . . ,Xk still lying on the nodes ui, . . . , and no other pebbles 
lying on t. Finally, A halts in a final state if and only if t [= . . . ,T/jb). 

The construction of A is easy for the atomic formulas. As an example, the 
automaton for x < y searches the input tree for the node u with pebble x, 
traverses the subtree with root u to see whether it contains the pebble y, 
and returns to the root. In the induction step it suffices to consider negation, 
conjunction, and universal quantification. For negation, just interchange final 
and non-final states (note that the automaton is deterministic and always- 
halting). For conjunction, just simulate the two given automata, one after the 
other. Now consider a formula </> = Vx : (f>^{x) and let A' be the automaton 
constructed for <^'. The automaton A for 0 traverses the input tree in pre- 
order and executes the following subroutine for every node u: it drops pebble 
X on u, walks to the root, simulates A', returns to pebble x, and lifts pebble 
X from u. 

This proves that all FO definable tree languages can be accepted by a 
deterministic tree-walking automaton with nested pebbles. Note that the 
automaton can be constructed in such a way that it uses k pebbles where k 
is the quantifier nesting depth of the formula. 

Next we prove that all p-twa languages axe regular. Let A be a p-tw 
automaton over the ranked alphabet S. We will show, by induction on the 
number of pebbles, that the trip T(A) can be computed by a tree- walking 
automaton M with MSO tests, i.e., T(M) = T(A). By the Proposition in 
Section 2 and the argument following it, this implies that all p-twa languages 
are MSO definable and hence regular. The basic idea is that a subcomputation 




Tree- Walking Pebble Automata 



81 



of A that corresponds to the life time of a pebble on node w, can be replaced 
by an MSO test on u. Without loss of generality we assume that the pebble 
names of A are 1, . . . ,n, and that at each moment of time pebbles 1,. . . ,i 
are lying on the input tree, for some 0 < i < n. The induction is on n. For 
n = 0, A is an ordinary tw automaton (and so we take M = A). Assume 
now that the result holds for n — 1. Consider a sub computation of A that 
corresponds to a life time of pebble 1. Obviously, during this life time, A 
behaves like a p-tw automaton with n — 1 pebbles on an input tree of which 
one node is “marked”. Let be the ranked alphabet E {a' \ cf ^ E), 
where each a' is a new (“marked”) symbol, of the same rank as a. Define 
A' to be the tw automaton with pebbles {2, . . . , n} that behaves in the same 
way as A, interpreting a node with label cr' as a node with label a that 
contains pebble 1; A' has initial state p and final state g, where p and q are 
the states of A at the start and end of the considered life time, respectively. 
By the induction hypothesis and the Proposition in Section 2, T(A') is MSO 
definable, i.e., there is a formula 0'(xi,X2) over E' such that T(A') = T(0'). 
Let <t>(x\,X 2 ,y) be the formula that is obtained from ^'{xi,X 2 ) by changing 
every atomic subformula \dhai{z) into the formula \dha{z) t\z — y and every 
atomic subformula \dha{z) into laha{z) A z ^y. Then, for all nodes ui,U 2 ,v 
of t € Ts, t \= (f>{ui,U 2 ,v) iff A can walk from ui in state p to U 2 in state q 
with pebble 1 on i; (without lifting pebble 1). Consequently, the considered 
subcomputation can be replaced by the MSO test <f>{x^x^x) on the current 
node. Thus, A is turned into a tw automaton with MSO tests that computes 
the same trip. For more details see Chapter 2 of [20] where this result is 
shown for the special case of 2- way automata on strings. 

This proves that all tree languages accepted by tree-walking automata 
with nested pebbles axe regular. We conjecture that they form a proper sub- 
class of the regular tree languages, and that the number of pebbles determines 
a proper hierarchy (by iterating the construction at the end of Section 4). 

6 Set-Pebbles 

It was shown in the previous section that the FO definable tree languages 
can be recognized by tree-walking automata that use their pebbles to im- 
plement the quantification of node variables. Thus, one naturally gets the 
idea that all mso definable tree languages (i.e., all regular tree languages) 
can be recognized by an automaton that uses a generalized type of pebble by 
which the quantification of node-set variables can be implemented. We will 
call such a generalized pebble a “set-pebble” . Let us define a (nondeterminis- 
tic) tree-walking automaton with set-pebhles to be a tree-walking automaton 
that carries a finite number of set-pebbles (with distinct colours) that it can 
drop, test, and lift. Dropping a set-pebble of a certain colour means that, 
nondeterministically, a pebble of that colour is dropped on any number of 
nodes of the input tree. Lifting a set-pebble of a certain colour means that 
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all pebbles of that colour are lifted from the tree. The automaton can test 
the colours of the pebbles that lie on the current node. Note that dropping 
and lifting are independent of the current node. Finally, as for pebbles, we 
require the life times of the set-pebbles to be nested. 

Let us now argue that the tree-walking automaton with set-pebbles ac- 
cepts exactly the regular tree languages. In one direction, it is well known 
(see [19]) that every regular tree language is MSO definable by a formula of 
the form (f) = 3Xi , . . . ,Xk : . . . ,Xk) where (f)' is locally first-order (in 

the sense that it does not contain 3X, VX, and x <y). Since, by Section 3, 
LFO formulas can be accepted by tw automata, it should be clear that, to 
implement 0, it suffices to use set-pebbles with colours Xi,. dropping 

them at the start of the computation, then simulating the tw automaton that 
implements (/>', and lifting them at the end of the computation. In the other 
direction we follow the proof in Section 5 that all p-twa languages are regular. 
As in that proof, consider a subcomputation of A that corresponds to a life 
time of the set-pebble with colour 1. Suppose that A drops the set-pebble in 
state p when it is at node and lifts it again in state q when it is at node v. 
This means that there is a set of nodes U (the nodes that are covered by the 
set-pebble) such that A walks from utov behaving like a tw automaton with 
n — 1 set-pebbles on an input tree t of which the nodes in U are “marked” . 
Thus, by the induction hypothesis and the Proposition in Section 2, there is 
a formula (j){x\,X 2 ^X) that models this behaviour (for x\ = u, x^ = v, and 
X = U). Hence t satisfies 'ip{u,v)^ where ^ is the formula 3X : </)(xi ,^ 2 , X). 
In other words, (t^u,v) is in the trip T(^). By the Proposition in Section 2 
there is a tw automaton M with MSO tests such that T{M) = T{ip). Conse- 
quently, the considered sub computation of A can be replaced by a simulation 
of M. This turns A into a tw automaton with MSO tests. 

We finally note that is not clear how one could define a deterministic 
version of the tree- walking automaton with set-pebbles. 
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Summary. We give a brief survey of the (un) decidable properties of multicounter 
machines. In particular, we present some of the strongest decidable results (con- 
cerning emptiness, containment, and equivalence) known to date regarding these 
machines. We also discuss some applications. 



1 Introduction 

A fundamental decision question concerning any class C of language recog- 
nizers is whether there exists an algorithm to decide the following question: 
given an arbitrary machine M in (7, is the language accepted by M empty? 
This is known as the emptiness problem (for C ) . Decidability (existence of an 
algorithm) of emptiness can lead to the decidability of other questions such as 
containment and equivalence (given arbitrary machines Mi and M 2 in C, is 
the language accepted by Mi contained (respectively, equal) to the language 
accepted by M 2 ?) if the languages defined by C are effectively closed under 
union and complementation. 

The simplest recognizers are the finite automata. It is well-known that 
all the different varieties of finite automata (one-way, two-way, etc.) are ef- 
fectively equivalent, and the class has decidable emptiness, containment, and 
equivalence (ECE, for short) problems. 

When the two-way finite automaton is augmented with a storage device, 
such as a counter, a pushdown stack or a Turing machine tape, the ECE 
problems become undecidable (no algorithms exist). In fact, it follows from a 
result in [12] that the emptiness problem is undecidable for two-way counter 
machines even over a unary input alphabet. In this paper, we look at various 
generalizations and restrictions of the counter machine model and give a brief 
survey of the (un)decidable properties of these machines. In particular, we 
present some of the strongest decidable results known to date concerning 
these machines. We also give some applications. 
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2 One-Counter Machines 

We begin with machines with only one counter. In what follows, 2DCM 
(IDCM) denotes a two-way (one-way) deterministic counter machine. 

We already know that the emptiness problem for 2DCM is undecidable. 
Suppose we restrict the operation of the 2DCM so that its input head makes 
only a fixed number of turns on the input tape (independent of the input). 
Even for the case when the machine makes only one input turn, we have: 

Theorem 1. The emptiness problem for 1-input turn 2DCM’s is undecid- 
able. 

Proof. We sketch the proof in [8]. We reduce the emptiness problem for Tur- 
ing machines to the emptiness for 1-turn 2DCM’s. Given an arbitrary Turing 
machine M, we can find two IDCM Mi and M2 and a homomorphism gi 
such that L(M) = g{L{Mi)CiL{M 2 )). Furthermore, we can find a INCM M3 
such that L{Ms) = L(M 2 )^. By expanding the input alphabet with markers 
to dictate the nondeterministic choices of M3, we can obtain a IDCM M4 and 
a homomorphism g 2 (which maps the markers to the null string and leaves 
the other symbols the same) such that L{Ms) = g 2 {L{M^)). We also modify 
Ml to include markers in its alphabet; the resulting machine M5 behaves like 
Ml and ignores markers in its input. 

From these constructions we have L(M) = gig 2 {L{M^)f\L{M/^)^), and it 
is now trivial to construct from M4 and M5 a 1-input reversal 2DCM M' such 
that L(M) = gig 2 {L{M')). Since the emptiness problem for Turing machines 
is undecidable, so is the emptiness problem for 1-input reversal 2DCM. ■ 

For one-way machines, we have: 

Theorem 2 ([13]). The equivalence (hence also the emptiness) problem for 
IDCM’s is decidable y but the containment problem is undecidable. 

Now suppose we restrict the 2D CM so that its counter is finite-reversal 
(i.e., the number of alternations between increasing and decreasing modes 
is at most a fixed number, independent of the input) but its input is unre- 
stricted. Such machines can accept fairly complex languages, even when the 
counter makes only 1-reversal. For example, such a machine can recognize 
the language consisting of strings of the form where i divides j. A 1- 
reversal counter machine stores j in the counter and makes sweeps over the 
a-segment, subtracting i fi:om the counter for each sweep. 

In fact, it can be shown that finite-reversal 2DCM’s can verify the validity 
of existential sentences in the first-order logic for integers with order and 
addition. 

In contrast to Theorem 1, we have: 

Theorem 3 ([14]). The ECE problems for finite-reversal 2D CM^s are decid- 
able. 
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We believe that the class of finite-reversal 2 DCM’s is one of the largest 
known classes of machines, which are a natural generalization of the two-way 
finite automaton, for which the ECE problems are decidable. 

Consider now a 2 DCM where the counter can become zero only at most a 
fixed number of times regardless of its input (call it finite-reset 2 D CM). These 
machines are powerful. For example, there is a finite-reset 2 DCM M that 
accepts the set S = > !}■ M first checks that its input is 

of the form # 0 ^^# • • • # 0 ^” and that n — x. This requires 1 reset of the 

counter. Next M verifies that yi > y2 by adding l-\-yi~y2 to its counter; M 
rejects if the counter resets to zero during this operation. Otherwise, y\ > y2 
and M adds 2 y 2 — yi to the counter so that its value now becomes 1 -h 2/2 . 
Note that the counter does not reset during this operation. M now checks 
that 2/2 > 2/3 and so on, until it has verified that yi > y2 > • • • > yn- Finally, 
M verifies that 2/1 = Vn- It is easy to see that the whole computation requires 
only 2 resets of the counter. 

It follows from the above remark that the emptiness problem for finite- 
reset counter machines is undecidable, because they can multiply and hence 
compute the value of any polynomial p{y, , ^2, . . . , (see [ 2 ] for the proof 
of an analogous result). However, over bounded languages, we can show that 
the emptiness problem is decidable. 

Definition 1 . A language L is bounded if it is a subset of • • • ajj, for some 
distinct symbols ai , . . . , a^; . 

Actually, the notion can be relaxed. The symbols need not be distinct, 
provided there are at most a fixed number of runs of symbols in the strings. 
For instance, a language C a* 02 aJ! 02 03 a* is also viewed as bounded since it 
has at most 6 runs of symbols. But (0*02)* is not bounded. In this paper, we 
use the relaxed definition. 

Each accepting computation of a finite-reset 2 D CM M can be divided into 
a finite number of phases at the times the counter value becomes zero. Since 
the language is bounded, it is easy to see that the counter makes a reversal 
only at the boundary points, i.e., the endmarkers or the first symbols of each 
type. The computation in each phase can then be rewritten in the form uv^w, 
such that the lengths of and w are bounded by some constant depending 
only M, and v starts and ends in the same state and on the same boundary 
point. Again, the number of different such v’s is bounded by a constant. So 
we can construct a finite-reversal 2 DCM M', whose input is padded with the 
net change to the counter value by each possible loop such that L{M) is 
empty iff L{M') is. Hence from Theorem 3 , we have: 

Theorem 4 . The ECE problems for finite-reset 2 DCM’s over bounded lan- 
guages are decidable. 

It is interesting to note that there are bounded languages that can be 
recognized by finite-reset 2 DCM’s but not by finite-reversal 2 DCM’s. One 
example is L = { 0 ® 1 ^ 2 ^ : a = n{b - c) for some n > 0 }. 
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3 Multi- Counter Machines 

When the input and counter of a one-counter machine are restricted, the 
decision problems become decidable. It is known that the ECE problems are 
decidable for counter machines with a finite-turn input and a finite-reversal 
counter. In fact, stronger results are known [8,9]. For nonnegative integers 
m, k and r, we define a finite-crossing finite-reversal multi-counter machine as 
a two-way finite automaton with input delimiters (end markers) , augmented 
with m counters such that on any input: (i) Any (possibly nondeterministic) 
computation leads to a halting state with the input head on the right de- 
limiter; (ii) No boundary between input symbols (including the delimiters) 
is crossed by the input head more than k times (note that the number of 
turns, i.e., changes in directions, the input head makes on the input may be 
unbounded); (iii) Each counter makes no more than r reversals. Note that 
each counter can be tested whether it is 0 or non-0 and incremented or decre- 
mented by 1. We count each alternation from increasing mode to decreasing 
mode or vice-versa as a reversal. We call such a machine a fc-crossing r- 
reversal m-counter machine. When m,k,r are not specified, we refer to the 
machine as finite-crossing finite-reversal multi-counter machine. Note that 
1-crossing corresponds to one-way input. 

Finite-crossing finite-reversal multi-counter machines are quite powerful. 
For example, a deterministic 5-crossing 1-reversal 1-counter machine M can 
accept the language over the alphabet {a, 6, c, d} consisting of all strings such 
that the sum of the lengths of all runs of c’s occurring between pairs of 
symbols a and b (in this order) equals the number of d’s. For example, M 
accepts the string ‘‘dachacaccbdd\ M operates in the following manner. It 
computes the sum in its counter by looking at the input and whenever it 
sees an a, it first checks that there is a matching b to the right and that all 
symbols in-between are c’s. It then moves left (to a), adding the length of the 
run of c’s to the counter. The process is repeated until the whole string has 
been examined. (So far, M crosses any boundary between two input symbols 
at most 3 times.) M then moves the input head from the right delimiter to 
the left delimiter and checks that the number of d’s is equal to the sum in the 
counter. Finally, the input head is moved to right delimiter and the machine 
accepts if and only if the string is in the language. Thus, M is 5-crossing, 
although its input head makes an unbounded number of (left-to-right and 
right-to-left) turns, i.e., it is not finite-turn. In fact, it can be shown that no 
deterministic finite-turn finite-reversal multi-counter machine can accept the 
language. 

As another example, let L be the language consisting of all strings 
such that x,y,z are pair-wise distinct binary strings. A nondeterministic one- 
way 1-reversal 3-counter machine M can accept L. M uses one counter to 
check that x is different from y^ a second counter to compare x and 2 ;, and a 
third counter to check that y is different from 2 :. To verify that x is different 
from y, M “guesses” a position of discrepancy (within the string x). It does 
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this by incrementing the first counter by 1 for every symbol it encounters 
while moving right on x, and nondeterministically terminating the counting 
at some point, guessing that a position of discrepancy has been reached. M 
records in it’s finite-control the symbol in that position. M uses the value in 
the counter to arrive at the same location within y where a discrepancy was 
guessed to occur. The second and third counters are used in a similar way to 
compare x with z and y with z. 

The following results are from [9] : 

Theorem 5. The emptiness problem for nondeterministic k- crossing 
r -reversal m-counter machines is decidable in nondeterministic {mkrXogn)- 
SPACE and in deterministic -TIME, where n is the length of the binary 
representation of the counter machine being tested and c a constant. 

Theorem 6. The containment and equivalence problems for deterministic 
k- crossing r -reversal m-counter machines are decidable in nondeterministic 
(mkr\ogn)-SPACE and in deterministic n^'^^'^-TIME, where n is the sum of 
the lengths of the binary representations of the two counter machines involved 
and c a constant. 

Theorem 6 fails for nondeterministic machines. In fact, the containment 
and equivalence problems are undecidable for nondeterministic one-way 1- 
reversal 1-counter machines [4]. However, when the machines accept “bound- 
ed” languages, the problems become decidable [8,9]. 

Theorem 7. Let k,r be fixed nonnegative integers. The containment and 
equivalence problems for nondeterministic k-crossing r-reversal m-counter 
machines accepting bounded languages are decidable in -TIME for some 

polynomial p, where n is the sum of the lengths of the binary representations 
of the machines involved. 

Theorem 8. Every nondeterministic finite- crossing finite-reversal multi- 
counter machine which accepts a bounded language can effectively be con- 
verted to an equivalent deterministic finite- crossing finite-reversal multi-coun- 
ter machine. The result also holds for the case of “one-way^’ instead of ^‘finite- 
crossing”. 

In the results above, the restriction that the input head is finite-crossing 
is crucial in view of the following result in [8]. 

Theorem 9. The emptiness problem for deterministic finite-reversal 2-coun- 
ter machines (with unrestricted input, i.e., not finite- crossing) accepting 
bounded languages is undecidable. 
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4 Some Applications 

We give some applications of the results of the previous sections. 

The first concerns finite-reversal two-way deterministic pushdown au- 
tomata (2DPDA’s) on bounded languages. A 2DPDA is finite-reversal if it 
makes at most a fixed number of alternations between ’’pushing” and ’’pop- 
ping”, independent of the input. 

Theorem 10. The emptiness problem for finite-reversal 2DFDA^s accepting 
hounded languages is decidable. 

The proof follows fi'om Theorem 3 and the following lemma. 

Lemma 1. Let M be a finite-reversal 2DPDA accepting a bounded language 
over a\. . .a*^. We can effectively construct a finite-reversal 2DCM machine 
M' (not necessarily accepting the same language) such that L{M) is empty 
iff L{M‘) is empty. 

Proof. We may assume, without loss of generality, that on every step, M 
pushes exactly 1 symbol on top of the stack, does not change the top of the 
stack, or pops exactly 1 symbol, i.e., M is not allowed to rewrite the top of 
the stack. In the discussion that follows, we assume Mis processing an input 
that is accepted, i.e., the computation is halting. 

A writing phase is a sequence of steps which starts with a push and the 
stack is never popped (i.e., the stack height does not decrease) during the 
sequence. A writing phase is periodic if there are strings u,v,w with v nonnull 
such that for the entire writing phase, the string written on the stack is of the 
form uv^w for some i (the multiplicity) , and the configuration of M (state, 
symbol, top of the stack) just before the first symbol of the first v is written 
is the same as the configuration just before the first symbol of the second v 
is written. Note that w is a> prefix of v. Clearly, a writing phase can only end 
when the input head reaches an endmarker or a boundary between the a^’s. 
A writing phase can only be followed by a popping of the stack (i.e. reversal) 
or another writing phase with possibly different triple (u,v,w). 

By enlarging the state set, we can easily modify M so that all writing 
phases are periodic. One can easily verify that because M is finite-reversal, 
there are at most a fixed number t of writing phases (in the computation), 
and t is effectively computable from the specification of M. 

We now describe the construction of M'. Let x = . . .a]f be the in- 

put to M. The input to M' is of the form: 2 /#Ci#C 2 # . . . #c^, where the 
Ci’s are unary strings; y is x but certain positions are marked with markers 
mi , m 2 , , mt. Note that a position of y can have 0, 1, ... at most t markers. 

M' simulates M on the segment y ignoring the markers. The c^’s are used 
to remember the counter values. Informally, every time the counter enters a 
new writing phase, the machine “records” the current value of the counter 
by checking the c^’s. 
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M' begins by simulating M on y (ignoring the markers). When a writing 
phase is entered, M' records the triple (wi, in its finite control and the 

multiplicity in the counter. Suppose M enters another writing phase. Then 
M' checks that the input head is on a symbol marked by marker mi ; hence 
M' can “remember” the input head position. (If it’s not marked mi, M' 
rejects.) Then it “records” the current value of the counter on the input by 
checking that the current value is equal to ci. (If it’s not, M' rejects.) M' 
restores the input head to the position marked mi and resets the counter 
to 0. It can then proceed with the simulation. Next time M' has to record 
the value of the counter, M' use the input marker m 2 and checks C 2 , while 
storing the triple {u 2 ,V 2 ,W 2 ) in its finite control, etc. Popping of the stack is 
easily simulated using the appropriate triple (u,v,w) and the counter value. 
Note that if in the simulation of a sequence of pops, the counter becomes 
0, M' must first retrieve the appropriate Ci (corresponding to the pushdown 
segment directly below the one that was just consumed) and restore it in the 
counter before it can continue with the simulation; retrieving and restoring 
the count in the counter requires the input head to leave the input position, 
but M' can remember the “new” position with a new “marker” . If after a 
sequence of pops M enters a new writing phase before the counter becomes 
0, the “residual” counter value is recorded as a new Ci like before. We leave 
the details to the reader. It is clear that M' is a finite-reversal 2DCM. I 

Since finite-reversal 2DPDA’s accepting bounded languages can be made 
halting [14], we have: 

Corollary 1. The containment and equivalence problems for finite-reversal 
2DPDA’s accepting hounded languages are decidable. 

Note that the above result (for equivalence) is a generalization over bound- 
ed languages of the well-known result [15] that the equivalence problem for 
one-way finite-reversal pushdown automata is decidable. 

It is known that 2DPDA’s are more powerful that 2DCM’s [16]. Consider 
the set of palindromes L = {x#x^ | x G {0, 1}* }. This language cannot be 
accepted by any finite-reversal 2DCM [1]. On the other hand, L can easily be 
accepted by a 2DPDA (in fact, a one-way DPDA) whose stack reverses once, 
and by a 2DCM with an unrestricted counter. It follows that finite-reversal 
2DCM’s are weaker than 2DCM’s and weaker than finite-reversal 2DPDA’s, 
over general (i.e., unbounded) languages. In contrast, we can show: 

Theorem 11. Over hounded languages, every finite-reversal 2DPDA can ef- 
fectively be converted to an equivalent finite-reversal 2DCM. 

Note that the above theorem implies Theorem 10 and Corollary 1 . The- 
orem 11 is best possible since: (1) the emptiness problem for 2DCM’s (and 
hence also for 2DPDA’s) over unary alphabet is undecidable [12]; (2) the 
emptiness problem for 2DPDA’s over unbounded inputs whose input head 
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makes at most one turn and whose stack makes at most one reversal is un- 
decidable [8]. It remains an interesting open question whether 2DCM’s and 
2DPDA’s over bounded languages are equivalent. 

The emptiness, equivalence and containment problems for finite-turn 
2D PDA’s (with unrestricted stack) over bounded languages are decidable. 
In fact there is a stronger result. One can augment the pushdown automata 
with counters. The following interesting result was shown in [8] (“N” stands 
for nondeterministic). 

Theorem 12. The following problems are decidable. 

1. The emptiness problem for one-way NPDA’s (INPDA’s) augmented with 

finite-reversal counters (but the stack is unrestricted). 

2. The emptiness, equivalence and containment problems for finite-tum 

2NPDA’s augmented with finite-reversal counters over bounded languages. 

Theorem 12 has some nice applications. An example is concerned with 
checking PCP (Post Correspondence Problem) - like properties. Given two 
n-tuples of nonnull words (ui , . . . , u^), {vi,. . .,Vn) and two integer functions 
f, g whose range is [1, n]. Define the following languages: 

• L = { X \ X = Uf(i ) . . - Uf(k) = Vg(i) • -'^gik) for some k }, 

• L' = { X I X = . . . Uf(^k) = '^g{i) • • • '^g{k) for some k, 

where ^f(l), . . . , g{k) is a permutation of /(I), . . . , f{k) }, 

Let C be a context-free language specified by a INPDA or a CFG. Then one 
can effectively construct INPDA’s augmented with finite-reversal counters to 
accept L, L', and L' D C. It follows that the properties defined by L, L', and 
V f\C for arbitrary {u\ , . . . , Un] and {v\, . . . ,Vn) and C are decidable. Note 
that these properties are almost like the PCP property which is undecidable. 

In part 1 of Theorem 12, the one-way machine has one unrestricted push- 
down and several finite-reversal counters. If instead there is only one un- 
restricted counter and one finite-reversal pushdown, the problem becomes 
undecidable: 

Theorem 13. The emptiness problem is undecidable for one-way determin- 
istic one-counter machines augmented with 1 -reversal pushdown (the counter 
is unrestricted). 

Theorem 13 follows from the fact that the emptiness problem is undecid- 
able for 1-turn 2DCM’s [8]. 

We now consider finite-reversal 2NPDAs over bounded languages. We 
shall show that the emptiness problem for this class is undecidable even 
when restricted to unary alphabets. We first make the following observation. 
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Observation: 

Given a single-tape TM M, we can effectively construct a two-counter ma- 
chine (without input tape) M' which simulates M on blank tape. M' is 
normalized in that its computation can be organized in phases starting with 
both counters zero. Each phase starts with one counter Cl decreasing its value 
until it becomes zero, while the other counter C2 increases its value (from 
zero). In the next phase, C2 decreases to zero while Cl increases from zero, 
etc. The construction of M' is a simple modification of Minsky’s construction 
(see, e.g., [10]). 

Theorem 14. The emptiness problem is undecidable for 

1. sweeping 1-reversal 2NPDA’s over unary languages. 

2. 1-turn 1-reversal 2DPDA’s over the language (O”^#)"^. 

Proof. To prove 1), let M be a normalized 2-counter machine as described 
in the Observation. We construct a 1-reversal 2DPDA M' which simulates 
M as follows. Given an input a^, M' first writes (#a^)* on the pushdown 
for some nondeterministically chosen i. Then M uses the pushdown to sim- 
ulate the decreasing counter and the input to simulate the other counter. 
The simulation of a counter by the pushdown can be made by just popping 
the stack. When the simulation of a phase of the 2-counter machine M has 
been completed, M' continues moving its input head (in the same direction) 
and popping its stack (thus preserving the difference between the counter 
values) until the input head reaches an endmarker. By exchanging the roles 
of the input head and pushdown stack (with respect to the counters they are 
simulating), the simulation can then be continued. Clearly, M' is sweeping, 
almost deterministic, and makes exactly one reversal on the stack. 

To prove 2), we use the fact that a 2-counter machine (without input) 
normalized as described in Observation above can simulate a TM on blank 
tape. So let M be such a 2-counter machine. We construct a 2DPDA M' 
which simulates M as follows. Given an input, M' copies the input on the 
pushdown and checks that it is of the form #0®^# . . . #0®* . (% is the left 
endmarker.) At the end of this process, the input and pushdown heads are 
on $. The simulation of M is done while M' moves its input head to the left 
and pops it stack towards %. M' simulates the 2-counter machine but when 
a phase of M is completed, M' continues moving its head and popping its 
stack until the input head and stack are on #. Then it simulates the next 
phase, etc. Note that the simulation can be carried out correctly if the i/s 
are large enough and it is not necessary that they are the same. I 

Another interesting application is in the theory of query languages for con- 
straint databases. Counter machines provide a new technique to the study 
of containment and equivalence (CE) of queries with linear constraints (over 
N, Z,Q, E). In [11], it is shown that the CE problems are decidable for con- 
junctive queries and for a subclass of first-order queries. In the following, we 
give a short discussion on the problems. 
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We fix the domain Z (the domain N is similar; Q, E are dealt with in 
[11]). A (database) schema is a set of predicates and a database is a mapping 
that maps each predicate to a finite subset of Z” of an appropriate arity. 
A conjunctive query over a schema a has the following form: “an5(x) <r- 

where n € N, are sequences of variables 

such that {z} C {^i, 9i’s are predicates in cr, and is a quantifier- 

free first-order formula involving linear constraints over Z . The answer to a 
conjunctive query Q on a database /, Q{I), is the result of applying the rule 
to I. A query Q is contained in another Q', <5 E Q', if for each database /, 
Q{I) C Q'{I). Two queries are equivalent if they contain each other. 

Theorem 15. [11] The CE problems of conjunctive queries are decidable. 

The proof consists of two steps. First, we show that the CE problems can 
be determined by examining only finite databases of a fixed size, i.e., the total 
number of elements occurring in the database (dependent on the queries). 

Lemma 2. For conjunctive queries Q,Q\ there is k£N (<cxsize of Q for 
some constant c) such that QQQ^ iff for all databases I of size <k, 
Q(/)CQ'(/). 

Proof. Since all relation names occur positively in a conjunctive query, the 
query is monotonic with respect to databases. The lemma follows directly. I 

For the second step, let Q,Q' be two conjunctive queries. We construct 
two bounded languages Lq and Lg/ that are accepted by nondeterministic 
one-way 1-reversal m-counter machines for some m € N (depending on the 
queries) such that Lq C Lq> if and only if Q C Q'. The construction is 
accomplished by Lemmas 3-5 in the following. 

The languages Lq and Lg/ are defined using a unary encoding of data- 
bases. The alphabet includes the symbols ci, ..., c^,+, -, $, where m is the 
largest arity of relations in the database. A tuple t = (ai,...,ak) C Z^ is 
encoded as ENC(t) = where Si is the sign of Oi and Vi the 

absolute value of ai. Let / be a database of the schema a. If is an 

enumeration of a relation p in /, the encoding of p is “ENC(ti)...ENC(t£)$”. 
The encoding of the database /, enc(/), is the concatenation of encodings of 
its relations. 

Let instk{cr) be the set of databases over a with at most k elements. We 
define Lg = { ENC(t) $ enc(/) | I G instk{cr),t G Q(I)}- Since k is fixed, Lg 
is a bounded language. It is easy to verify the following. 

Lemma 3. Let Q,Q' be two conjunctive queries over a schema a and k GN 
greater than the product of the size of Q and the largest arity of predicates in 
a. Then Lq C Lg, if and only if Q Q Q' . 

We now show that Lg can be accepted by a nondeterministic one-way 1- 
reversal m-counter machine. Since databases have at most k elements, there 
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are at most assignments, where n is the size of Q. Let ai, ...,akn be all 
assignments. Let Lq be the sub-language of Lq where the encoded answer 
tuple can be produced by a{. We first need the following technical lemma. 

Let be a formula with free variables xi,...,Xk and linear constraints 
(e.g., (5xi — 3x2 — 4x3 4- 6 < 2 x 4 ) A -i(— 2xi 4- 7xs = X 4 )). Let ai,. . . ,ak be 
distinct symbols and L{ip) the language consisting of strings , 

where each Si is the sign (-h or -) and each £ N such that the formula (p 
is true under the assignment X{ SiV{ for each 1 <i < k. 

Lemma 4. For each quantifier-free formula p of size n we can effectively 
construct a deterministic one-way f{n) -reversal multi-counter machine 
that accepts L{p>), where both f(n) and the size of are polynomials in n. 

Proof. The states (finite control) of remember all coefficients including 
the signs and p. For each variable x, if a is the largest absolute value of its 
coefficients, has the following counters: Ca;,o, Ca;,i, ..., c^. |-ioga] ? where 
Cx^i will store the value 2* x x. Initially Cx,o gets the value from the (one-way) 
input. For each 0 < i < [log a] — 1, the value of Cx, 2 +i is computed from Cx,i 
using as an auxiliary counter. The value of Ca;,j is restored after computing 
the value of Cx,i+i. For each (occurrence of a) term t, includes a counter ct 
that will hold the absolute value for the term (the sign is remembered in the 
control). The (absolute) values of the counters for terms “ax” are computed 
fi:om the counters Cx^i, 0 < i < [log|a|], possibly using the auxiliary space 
c^. The values of the counters Cx/s are again restored after their use. We 
note that the binary representation of |a| and the sign are remembered in 
the finite control. The values of the counters for the other terms are then 
computed based on the syntax of the terms. Finally, performs comparison 
tests for =, <, and <, and then computes truth value of the formula p 
with Boolean operations. Clearly has polynomially many counters and a 
polynomial size in the size of (p. As for the number of reversals, we note that 
each “doubling” in computing Cx^ and each addition/ subtraction operation 
may need a couple of reversals. It follows that the total number of reversals 
is polynomial in the size of p. ■ 

Lemma 5. For each conjunctive query Q of size n, each k £ N, and each 
assignment a, Lq^ is accepted by a deterministic one-way 1-reversal f{n^k)- 
counter machine M, where f is a function polynomial in both n and k. The 
size of M is also polynomial in n and k. 

Proof For the assignment a, we construct first a deterministic one-way /- 
reversal multi-counter machine M' which, on an input “ENC(t)$ENC(/)”, ver- 
ifies if t G Q{I) under the assignment a, where / is a polynomial in both n 
and k. The verification involves (1) checking if the tuple for each atom is 
in the corresponding relation of /, which can be done in the straightforward 
manner, and (2) checking if the formula is satisfied, which can be done by a 
one-way, polynomial-size, polynomial-reversal counter machine by Lemma 4. 
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Therefore, M' can be constructed, and it is easy to obtain an equivalent 
1-reversal machine M. I 

To establish Theorem 15, let Q, Q' be two conjunctive queries and k 
such that Q Q Q' coincides with their containment over databases with 
at most /u-elements. Let ai,...,ak^ be all possible assignments. For each 
I <i < k^, let Mi (resp. M-) be the counter machine constructed in Lemma 5 
for Q (resp. Q'). Then, ^^d = Ui<i<A;n It 

follows that both Lq and Lq, are accepted by nondeterministic one-way 
finite-reversal multi-counter machines. By Theorem 7, the problem of deter- 
mining Q nQ' is decidable and so is the equivalence problem. 

The above decision procedure has an exponential space complexity in the 
size of queries. With an improvement in the last step, it can be shown to be in 
polynomial space [11]. In [11], the decidability results were further extended 
to a larger class of conjunctive queries with linear arithmetic over integers or 
rational numbers, and to a syntactically restricted class of first-order queries 
over bounded degree databases. 
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On the Equivalence 

of Finite Substitutions and Transducers 



Juhani Karhumaki and Leonid P. Lisovik* 



Summary. This paper discusses on several variants of a fascinating problem of 
deciding whether two finite substitutions are equivalent on a regular language, as 
well as its relations to the equivalence problems of sequential transducers. Among 
other things it is proved to be decidable whether for a regular language L and 
two substitutions (p and the latter one being a prefix substitution, the relation 
(f{w) C ^p{w) holds for aJl w in L. 



1 Introduction 

Equivalence problems of automata have played a crucial role in the theory of 
automata from its very beginning. Two such problems, namely the equiva- 
lence problems of deterministic multitape automata and deterministic push- 
down automata, were for several decades among the most important open 
problems of the theory, until they were solved in the 90’s, cf. [10] and [21], 
respectively. 

In its pure form the equivalence problem of automata asks to decide 
whether two automata of a certain type define the same language or rela- 
tion. An interesting variant of this problem was introduced by Culik II and 
Salomaa in [5], where they proposed to study so-called morphic equivalence 
problem for languages, i.e. a problem of deciding whether two morphisms are 
equivalent on a given language. Here being equivalent means that the mor- 
phisms map each word of the language to the same word. The celebrated 
DOL problem is an example of such a problem, cf. [4]. 

A natural generalization of the latter problem is to consider more gen- 
eral mappings than morphisms. Let us refer such decision problems to as 
the equivalence problems of mappings on languages. As an example of arti- 
cles considering these problems we mention [15], where it was shown that 
the equivalence of mappings of the form h~^g, with h and g morphisms, is 
decidable on regular languages while that of mappings of the form gh~~^ is 
undecidable. 



This work was initiated when this author visited Turku University under the 
grant 14047 of the Academy of Finland. 
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This paper concentrates on a particular instance of the above problem, 
namely on the equivalence of finite substitutions on regular languages. Sur- 
prisingly, this problem was shown to be undecidable in [18]. Our goals here 
are as follows. First, we recall connections of the above result with the equiv- 
alence problems of transducers, cf. also [22]. Second, we prove some new 
results, mainly decidability ones, for certain restricted variants of the above 
problem. These, we believe, sharpen the borderline between the decidability 
and undecidability. And finally, we consider the remaining major open prob- 
lem, namely the problem of deciding whether two finite substitutions are 
equivalent on the language ab*c, and give both a decidable and undecidable 
variant of this problem. 

In details this paper is organized as follows. 

In Section 2 we fix our terminology and recall several major results related 
to our problems. In Section 3 we consider so-called prefix substitutions, and 
prove, as our most complicated technical result, that not only the equivalence 
but also the (pointwise) inclusion of such substitutions on regular languages 
is decidable. In Section 4 we translate our results to transducers. Finally, in 
Section 5 we discuss on the most challenging open problem, the equivalence 
of finite substitutions on the language ab*c. 

2 Preliminaries 

In this section we fix our terminology, as well as recall a few important 
results we shall need in our considerations. We refer to [12] and [1] as general 
references. 

We denote by U* and the free semigroup, the free monoid and 
the free group generated by an alphabet U, respectively. Elements of U* are 
called words and 1 denotes the neutral element of U*, i.e. the empty word. 
The length of a word w is denoted by \w\. For two words u and v the word 
u is prefix of u, in symbols u < v, ii there exists a word t such that v = ut. 
We use the notation Pref(v) to denote the set of all prefixes of v. 

We say that a subset X C is a prefix set, or briefly a prefix, if none of 
its words is a prefix of another of its words. Consequently, 1 is in a prefix X 
iff X = {1}. Further we say that X C X* has a bounded deciphering delay if 
there exists a natural number p > 0 such that 

uX^E^ n i;X* = 0 for alU,t; E X with u^v. 

Obviously, X with a bounded deciphering delay is a code. 

A finite substitution is a morphism ip: E* V{A*), where V{A*) denotes 
the monoid of finite subsets of zi* under the operation of the product of 
subsets of A*. We call ip 1-free if 1 ^ ip{a) and prefix if ip{a) is a prefix set for 
all a € X. Consequently, a prefix substitution need not be 1-free, since it may 
be that ip{a) — {1} for some a £ E. Similarly, we say that ip has a bounded 
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deciphering delay if (p(a) has such for each a G U. Finally, we say that two 
substitutions U* V{A*) are equivalent on a language L C U* 

p(w) = il){w) for all w G L. 

Now, we are ready to formulate our crucial problems. 

Problem 1. Is it decidable for two finite substitutions : S* -> V{A*) 
and a regular language L C E* whether or not (f and 'ip are equivalent on L? 

Problem 2. Is it decidable for two finite substitutions p^'ip : {a,6, c}* 
V{A*) whether or not (p and 'ip are equivalent on ab'^c? 

Of course. Problem 2 is a very restricted variant of Problem 1 but we prefer 
to formulate it separately due to its importance. Note that the deterministic 
variant of Problem 1, i.e. when ip and 'ip are morphisms, in other words when 
sets ip{a) and ip{a) are singletons, is easily seen to be decidable. 

Next we fix our terminology on transducers, or more precisely on sequen- 
tial transducers. A sequential transducer'll a sixtuple M = (Q, E, A, T, qo, F), 
where Q is a finite set of states, E and A are finite input and output alpha- 
bets, respectively, T C Q x Z* x Zi* x Q is a set of transitions, qo is an initial 
state and F is a set of final states. The relation defined or computed by M is 
denoted by IZ(Ai) C E* x A*, for details cf.[l]. The underlying automaton of 
M is obtained from it by omitting the third components in the transitions. 

We shall need some special types of sequential transducers. We say that 
a sequential transducer M 

- is 1-free if, for each (p, a,v,q) G T, the word v 1] 

- is input deterministic if the underlying automaton is deterministic; 

- satisfies the prefix condition if, for each p G Q and a G E we have 

{p->a,v,q),{p,a,v' ,q') gT ^ v ^v' and F ^ v. 

The above definitions deserve a few comments. First the prefix condition 
implies (but is not equivalent to) the property: the set of outputs associated 
to each triple (p,a,q), i.e. 

Op,a,q = {v\{p, a,v,q)r\T ^ 0} (1) 

is a prefix set. For input deterministic transducers the latter condition is 
equivalent to the prefix property. Finally, a transducer with the prefix prop- 
erty need not be 1-free, but cannot contain transitions (p, a,l,q) and (p, a, 1, 
q') with q^q' . 

The following connection between two types of problems we are consider- 
ing was noticed in [3] . 

Theorem 1. The equivalence problem for input deterministic sequential 
transducers is decidable if and only if the equivalence of finite substitutions 
on regular languages is decidable. 
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The proof of Theorem 1 is based on the fact that the set of accepting com- 
putations of an automaton is a regular language. Moreover, the equivalence 
in Theorem 1 remains if the sets (1) of the input deterministic transducers 
are of the same type, for example prefixes, as the images of letters in the 
finite substitutions. 

We conclude this section by recalling two remarkable results related to 
our problems, and used later in our considerations. The first one, proved 
simultaneously in [13] and [17], extended in the very nontrivial way the basic 
undecidability result of sequential transducers established in [6]. Recently 
this result was further extended in [19] by considering also the number of 
states as a parameter. 

Theorem 2. The equivalence problem for 1-free sequential transducers hav- 
ing a unary output (or input) alphabet is undecidable. 

The other result proved in [18], cf. also [8] or [9] for a more detailed proof, 
gives a surprising answer to our Problem 1. It also gives a partial answer why 
we introduced Problem 2 as a separate problem. 

Theorem 3. It is undecidable whether two finite 1-free substitutions : 
{a, 6, c, d}* V{A*) are equivalent on the language a{b,c}*d. 

Using Theorem 1 the above can be interpreted as an undecidability result 
of sequential transducers. 

Corollary 1. The equivalence problem for 1-free input deterministic sequen- 
tial tranducers is undecidable. 

3 Problems on finite substitutions 

In this section we state two decidability results for a special type of substi- 
tutions, namely prefix substitutions. Our first result is noticed in [14], and 
should be compared to Theorem 3. 

Theorem 4. It is decidable whether two prefix substitutions are equivalent 
on a given regular language. 

In order to compare the proof techniques of different results of this paper 
we recall basic ideas of the proof of Theorem 4. It is based on three facts: 
The pumping lemma for regular languages, the freeness of the submonoid of 
the monoid V{A*) constituting of finite prefix languages, and the following 
implication of equations over free monoids: 

{ uv — u'v' 

uxv = u'x'v' uxyv = u'x'y'v' . 

uyv = u'y'v' 
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The second condition was first noticed in [20] while the third one is a simple 
exercise on word equations. 

In our second result we consider instead of the equivalence the inclusion 
of two substitutions (p and ^ on a regular language L, i.e. the relation (f{w) C 
'(p{w) for al\w£L. It seems that we need a completely new method than the 
one used in Theorem 4 to prove the decidability. 

Theorem 5. It is decidable whether for a given regular language L and two 
Ufree prefix substitutions (p and 'll) the relation (p{w) C 'ip(w) holds for all w 
in L. 

Proof Let (p and ip be 1-free prefix substitutions defined on U* and L C E* 
a regular language recognized by an automaton Al> The problem of the 
theorem is reduced to the emptiness problem of finite automaton, i.e. we 
construct a nondeterministic finite automaton A such that 

L{A) =0 if and only if p{w) C ip{w) for dl\ w £ L. (*) 

We may assume that the automaton Al = {Q,E,6,qo,F) is deterministic 
and reduced. Then define the constant 

t = max{min{|i;| | S{q,v) G F}}. 
q€Q 

Consequently, from any state of there exists a word of length at most t 
leading to a final state. Two other constants are defined as 

mi = max{max{|vl | v G p(ci)}}, 

m 2 = max{max{|v| | v G ipia)}}- 

Now, we construct the nondeterministic A as follows. 

Consider a constant N (fixed later). The automaton A 

(i) reads nondeterministically N symbols, i.e. a word w = a\ 
checks that w is a> prefix of a word in L; 

(ii) quesses nondeterministically a word u = ui .. .un, with U{ 
z = 1 , . . . , A^, and; 

(iii) checks that xp can respond to the above choice of (n), i.e. 
words Vi G ip{ai), for ^ = 1, . . . , iV (= m), such that 

vj = vi .. . vn 7 = u for some word 7, (1) 

or finds an ni < A/’ and words vi, . . . , such that 

■= v\ . . .Vni^ = u for some 7 G Pref((/?(a ni+l))- (2) 



m , m Cl jv j and 
G p{ai) for 
either finds 
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Now, a crucial observation is that, for a fixed w and u, either (1) or (2), 
but not both, can hold and, moreover, the i^^-sequence is unique. This is due 
to the fact that the sets 'tp{ai) are prefix sets. Further, if neither (1) nor (2) in 
(Hi) cannot be satisfied, then, by the choice of w and u, w can be extended 
to a word wwi such that ip{wwi) 2 that is the automaton A can 

accept its input. 

So it remains to analyse how the automaton A is constructed if (1) or (2) 
is satisfied. 

Consider first the case (1). Now, the automaton A checks whether 

l7l > m^t, (3) 

and if the answer is affirmative, then, as above, we can extend the word w to 
a word ww' in L such that ip{ww') ^ and so the automaton A can 

accept its input. Intuitively, this means that is too much behind in order 
to respond the choice of u done in (n). 

So assume the case (2) in (Hi). Now, the automaton A checks whether 

N — ni > mit, (4) 

and if the answer is affirmative, then again, but due to a different reason, 
namely because ^ is 1-free, we can extend it; to a word ww' in L such that 
2 'ip{ww'), and so the automaton A accepts its input. Intuitively, 
this case means that i/? is too much ahead in order to respond to a minimal 
continuation of w. 

If the answers to (3) and (4) are negative, i.e. the automaton A did not 
accept anything yet, then it remembers the word 7 (from (1) or (2)) and the 
number AT — m, and starts a new round of actions the automaton A 

reads new N symbols, say a'^^, . . . , chooses new N words, say u[ G 
for i = 1, . . . , AT, and the v[ words as in (Hi). The differences from the step 
described above are that now the new w, say w', is 

= ^u[ . . 

and that the new ni, say satisfies 

ni + n[ < 2AT, 

and accordingly (4) gets the form 



2N — ni — n[ > mit. 

The word 7' is defined as in the basic step. 

During the second round the information - ni) is changed to (7', 
2N -ui -n'l), where I7I, |7'| < m^t and N -m,2N -ni-n[ < mit. Hence, 
the procedure goes into a cycle. 

To complete the description of the construction of A we have to analyse 
what happens when a prefix of the n-letter word w in (ii) corresponds an 
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element of L. In such a situation the automaton A has to check that 'ip 
indeed can respond to the choice of the values of selected in {ii). 

Consider in more details the situations when the word in L is of the form 
ww* with |u;| = N and \w'\ < N. Say w — ai . . . aNCi'^^i • • • Now, after 

reading w the automaton A remembers the word 7 and the number N — ni 
such that, for the values ui € for i = there exists values 

Vi E for i = 1 , . . . , ni, such that 

UI...UN =Vi...Vni^. 

Now, the automaton A guesses nondeterministically the values u[ £ 
for i = N 1, ... j, and checks if for this choice there exist words 
e for z = rii + 1 , . . . , AT + j such that 

7Uiv+l • • = Vni+1 • ■ 

If the answer to this check is no, then the automata accepts the input, oth- 
erwise not. 

The value of N can be chosen rather freely. Clearly, everything works if 
it would be larger than any of the important constants t, tmi and tm^, say 
N = (2mi 4- 2 m 2 + 2)t. 

Now, we have completed the description of the automaton satisfying (*), 
and also the proof of the theorem. □ 

The above result deserves two comments. First, no special properties of 
the substitution ip are needed, it need not be 1-free nor prefix. So the result 
can be generalized accordingly. Second, and more importantly, also need 
not be 1 -free: 

Theorem 6 . It is decidable for a given regular language L and two prefix 
substitutions ip and ip whether the relation ip{w) C ip{w) holds for all w in L. 

Proof. In the proof of Theorem 5 the 1-freeness of ip was needed only in the 
case when ip was ahead, i.e. in the case corresponding (2) of (Hi). Here the 
essential point was when the relation N — ni > mit was satisfied. From that 
we concluded that ip{w) 2 'tpi'w). Now, the same conclusion can be drawn if, 
instead of the above inequality, the inequality 

p > mit 

is satisfied, where p is the number of those letters a of the suffix of the 
input word aia 2 . . .ajN for which ip{a) 1. In the proof of Theorem 5 the 
automaton A remembered the previous N block of letters, and hence also the 
required suflSx of length N — ni of the input word read so far. In the current 
case the automaton have to remember a sparse subword of length p. 
Otherwise the changes are very straightforward. □ 
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4 Problems on transducers 

In this section we modify the results of Section 3 for sequential transducers. 
We prove three decidability results each of those using completely different 
techniques. Actually, these results form a sequence of proper extensions of 
each other. 

From Theorems 1 and 4, cf. also the discussion before Theorem 1, we 
directly derive: 

Theorem 7. The equivalence problem for input deterministic sequential 
transducers satisfying the prefix condition is decidable. 

We recall that the transducers considered in Theorem 7 are those input 
deterministic sequential transducers for which the sets 

Op,a,q = {v\{p,a,v,q) e T}, for p,q £ Q,a G S, 

are prefix sets. In this formulation we cannot omit the input determimism, 
since clearly for any sequential transducer we can construct equivalent one 
satisfying the condition that the sets Op^a,q ^tre singletons, and thus prefix 
sets. 

On the other hand. Theorem 7 can be extended to: 

Theorem 8. The equivalence problem for sequential transducers satisfying 
the prefix condition is decidable. 

Proof. Let Ad be a sequential transducer satisfying the prefix condition. Then 
the relation 1Z{M) defined by M is accepted by a deterministic 2-tape au- 
tomaton. Indeed, such an automaton A is constructed as follows: It reads in 
its first tape the input word, and in the second tape the output word. When 
M is in its state p so is A, and it reads a symbol, say a, from its first tape, 
and moves either to state q if (p, a, l,g) is a transition of Ad, or moves to 
the state Pa where it reads the second tape until it finds a word v such that 
{p,a,v,q) is a transition of Ad. Then A moves to state q and again reads a 
symbol from the first tape. 

The above explains the basic simulation step, and A indeed is determin- 
istic due to the prefix condition of Ad. The other details of the simulation 
are straightforward. 

Now, Theorem 8 follows from the decidability of the equivalence problem 
for 2-tape deterministic automata, cf. [2] or [10]. □ 

Our method of Section 3 allows to extend Theorem 8. 

Theorem 9. The inclusion problem for sequential transducers satisfying the 
prefix condition is decidable. 
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Proof. The proof of Theorem 5, as well cis that of Theorem 6, can be modified 
for Theorem 9. Let Mi and M 2 be two transducers of Theorem 9. Then a 
finite nondeterministic automaton A is constructed such that 

L{A) =0 if and only if HiMi) C 1Z{M2)’ 

The main difference from the proof of Theorem 5 is that now A has to 
remember also the corresponding states of M\ and Ad 2 . Moreover, the prefix 
property of Ad 2 guaranties that, for a given input word w = a\ .. .on and 
the t/-sequence, the ^-sequence defined in (in) is unique. □ 

We conclude this section with two remarks. First, as in Theorem 6, the 
transducer Adi can be assumed to an arbitrary sequential transducer. Second, 
Theorem 9 can not be deduced using the method of Theorem 8, since the 
inclusion problems for deterministic 2-tape automata is undecidable, cf. e.g. 
[10]. 

5 Finite substitutions on a6*c 

In this section we consider the most challenging open problem of the topic 
of this paper, namely the problem of deciding whether two finite substitu- 
tions are equivalent on the language ab*c. We introduce two variants of this 
problem, one being decidable and the other undecidable. 

This can be seen as an indication of the hardness of this problem. Fur- 
ther evidence of that was proved already in [16] where it was shown that 
the language L = ab*c does not have a finite test set with respect to finite 
substitutions, i.e. finite subset F of L such that to test the equivalence of two 
finite substitutions on L it is enough to test that on F, cf. [11]. The proof of 
Theorem 4 shows that for prefix substitutions such a test set exists. 

Our first result is a decidability one. Its proof uses two-way finite automata 
with a finite number of counters. Such an automaton is called reversal bounded 
if there is a constant t such that on any computation the automaton changes 
the direction of the head and the directions of the use of the counters at most 
t times. A computation of such an automaton is stack increasing (resp. stack 
decreasing) if the length of the stack is never decreased (resp. increased). 
Finally, a configuration of such an automaton, with two stacks, on input w 
is completely determined by the quadruple 

{q,w,m,n2) 

where g is a state, ni and ri 2 are the contents of the stacks, and 4- points the 
position of the head on input w. 

Theorem 10. It is decidable whether two finite substitutions with bounded 
deciphering delay are equivalent on the language ab*c. 
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Proof. The problem is reduced to the emptiness problem of reversal bounded 
two-way nondeterministic counter automata, which was shown to be decid- 
able in [7]. 

Let cp, {a, 6, c}* -> V{A*) be finite substitutions with a bounded deci- 
phering delay. We may assume that each ip{a) and ip{a) (resp. ip{c) and ^(c)) 
starts with a special symbol, say 4 (resp. *$'), which does not occur anywhere 
else in the images. 

The following claims constitute the core of the proof. 

Claim 1. There exists a one-way nondeterministic counter automaton Mi, 
with i and t initial and final states, respectively, such that Mi has a stack 
increasing computation 



if and only if 



from {i,i^wl^,0) to 



£ (p{ab^c). 

Claim 2. There exists a reversal hounded two-way deterministic 2- counter 
automaton M 2 , with i and t initial and final states, respectively, such that 
M 2 has a computation 



if and only if 



from (z,4w^‘$',fc,0) to (t,4^*^,0,0) 



( 1 ) 



4^^*$' ^ ^(a6^c). 

Proofs of the Claims. The proof of Claim 1 is very obvious, and actually 
does not require any assumption on (p. The automaton Mi constructed 
according to the following ideas. First, M 2 copies k to the second counter. 
Then, assuming that the elements 'ip{a) are lexicographically ordered. Mi 
chooses the first element ai of ip{a), and checks whether £ ai^(a^c). 
This can be done deterministically since ^ has a bounded deciphering delay. 
If the answer is no, the stacks are regenerated and the procedure is continued 
with the second element of 'ip{a), and so on. If the answer to the above tests 
is always no, then Mi goes to the required configuration of (1). Clearly, Mi 
does several, but only a bounded number of reversals on the input, as well 
as on the counters. So the claims are proved. □ 

Now, it is not difficult - using automata from Claims 1 and 2 - to construct 
a reversal bounded two-way nondeterministic counter automaton M such 
that 



L{M) = 0 if and only if (p{ab^c) C 'ip{ab^c) for all k >0. 
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Indeed, the automaton of Claim 1 is used to create a number k such that 
(^ 1 $’ = (f(ab^c), and the automaton from Claim 2 is used to determine 
whether ^ can respond, i.e. whether € 'ip{ab^c). Moreover, L{M) ^ 0 if 
and only if jp is not able to respond, i.e. if and only if <^{ab^c) ^ 'fp{ab^c) for 
some k. 

Now, the result follows when we check the inclusions ip(ab*c) C 'ip{ab*c) 
and 'ip{ab*c) C (^(a6*c). □ 

As we already pointed out, we actually proved a stronger result than 
Theorem 10, namely the decidability of the condition 

(p{ab^c) C ip{ab^c), for all A: > 0, 

where cp is arbitrary and ip has a bounded deciphering delay. In particular, 
in the formulation of Theorem 10 it is not necessary to assume that p and ip 
has a bounded deciphering delay to the same direction. 

Our second result of this section is an undecidability result, although 
rather special one. Now we consider finite substitutions from a free monoid 
into the free group that is morphisms v?: -> 

Theorem 11. It is undecidable whether two finite substitutions ip^ip: U* ^ 
are equivalent on the language ab*c modulo a free submonoid of 
i.e. whether the following holds for a free submonoid F* of A^*} : 

ifiab^c) nr* = iPiab^c) D F* for all k>0. 

Proof. The result follows straightforwardly from Theorem 2. Let 



— (Qj {(^}’>A,T,qo,F) 

be a sequential transducer with a unary input alphabet. We define a finite 
substitution 

ipM- {a,h,cY At^) 

by setting 

VM{a) = 9 o ^ 

VM {b) = fyuq~^ if (p,b,v,q) £ T, 
iPm(c) = F. 

Now, for two sequential transducers M and M' we clearly have: 

M and M' are equivalent 



if and only if 



n Zi* = ^ A: > 0. 



This completes the proof. □ 
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Summary. As an alternative to the two classical proofe for complementation of 
Biichi automata, due to Biichi himself and to McNaughton, we outline a third 
approach, based on stratified alternating automata with a “weak” acceptance con- 
dition. Building on work by Muller, Saoudi, Schupp (1986) and Kupferman and 
Vardi (1997), we present a streamlined version of this complementation proof. An 
essential point is a determinacy result on infinite games with a weak winning con- 
dition. In a unifying logical setting, the three approaches are shown to correspond 
to three different types of second-order definitions of a;-languages. 



1 Introduction 

In his seminal paper [Bii62], Biichi introduced a framework for defining sets of 
sequences, today called the framework of Biichi automata, and showed that 
the class of properties definable therein is closed under complementation. 
This result Wcts the key to establish a bridge between a;-automata theory 
and monadic second-order logic over infinite strings, which in turn opened 
a new chapter of automata theory with interesting applications in logic and 
computer science. Even after forty years this chapter is far from being closed, 
and it is worthwhile to reconsider the beginnings. 

There are two “classical” approaches to the complementation of Biichi 
automata. The first, as found by Biichi himself, stays in the framework of 
Biichi automata and provides a transformation of a nondeterministic Biichi 
automaton into such an automaton for the complement language. The second 
proof, due to McNaughton [McN66] and later sharpened by Safra [Sa88], 
involves a transformation to deterministic automata with a more general 
acceptance condition, the Muller condition [Mu63], and uses the fact that for 
deterministic Muller automata the complementation step is obvious. Both 
approaches involve nontrivial arguments: In the first case, a combinatorial 
result is applied (Ramsey’s Theorem B [Ra29]), and in the second a very 
intricate automaton construction is required. 

In the present paper, we expose a third proof strategy which so far did 
not attract much attention in the literature but has some advantages. In- 
stead of reducing the nondeterminism of Biichi automata to determinism 
with a more complex acceptance condition, this third approach is based on 
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a more general transition mode than nondeterminism, namely alternation, 
but at the same time the acceptance condition is made simpler: Instead of 
Biichi acceptance the so-called “weak acceptance” condition is used. As in the 
deterministic case, the weak acceptance condition is closed under negation, 
whence complementation is relatively easy. Moreover, the task of connecting 
Biichi automata to weak alternating automata is not as complex as to show 
that Biichi automata and deterministic automata are equivalent. 

The idea of weak alternating automata is due to Muller, Saoudi, and 
Schupp ([MSS86], [MS87], [MSS88]). However, in their work they emphasize 
complexity issues (especially regarding program logics and temporal logics) 
and not so much a reconsideration of the complementation problem. Recently, 
Vardi and Kupferman [KV97] have taken up the approach and supplied a self- 
contained complementation proof for Biichi automata. Their proof strategy 
does not make use, however, of the duality phenomenon which is character- 
istic for alternating automata. In the subsequent sections we outline such 
a more symmetric proof. The key ingredient is a result on determinacy of 
infinite games with a weak winning condition. An advantage of this proof 
architecture is its composition from rather elementary, easily verified “mod- 
ules”. Moreover, some conceptual points are clarified. For example, one sees 
that for defining regular cj-languages by automata, the use of liveness condi- 
tions (such as the Biichi acceptance condition) can be avoided, if one works 
with alternating automata. The complementation proof via infinite games 
also sheds some light on the relation between automata on infinite words 
and automata on infinite trees. In the game-theoretic framework, the proofs 
of complementation for cj-automata and for tree automata can be compared 
via the respective determinacy results. For Biichi automata complementation 
we shall need only a very simple determinacy proof based on a reachability 
analysis, whereas tree automata complementation requires the more com- 
plicated determinacy proof for parity games (cf. [Th97]). This pinpoints a 
characteristic difference between cj-automata theory and tree automata the- 
ory. 

This paper provides an introduction to results obtained in collaboration 
with C. Loding (see his diploma thesis [L698]) on alternating cj-automata. 
We confine ourselves to the use of weak alternating automata in the com- 
plementation proof, mentioning only briefiy how the transformation of Biichi 
automata into this model and conversely works, and leaving aside complexity 
issues and further applications of alternating automata. A joint paper with 
a more detailed exposition and further results is in preparation. 

2 Review of the classical proofs 

A Biichi automaton over the alphabet A is a finite automaton of the form 
A = (Q^A,qo,A,F) with finite set Q of states, initial state qo, transition 
relation A C Q x AxQ, and a set F C Q of final states. It accepts an cj-word 
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a = a(0)a(l) . . . from if there is a run p = p(0)p(l) . . . from with 
p(0) = go {p{i) ^ p{^ + 1)) ^ ^ for i > 0, which is Biichi accepting^ 
i.e. such that p{i) € F for infinitely many i. Formally, we write 

(*) 3p (p(0) = go A Vz((p(«),a(2),a(i + 1)) € A > i p{j) £ F) 

The (j-language L{A) recognized by A consists of all o;-words a for which (*) 
holds. 

In the logical classification of quantifier alternation hierarchies, one calls 
(*) a Z*! -formula, referring to the existential sequence quantifier in front 
(which in an arithmetical setting is captured by a tuple of existential second- 
order quantifiers ranging over sets of natural numbers). We shall also take 
into account the logical status of the acceptance condition, which in the case 
above is called a -condition, referring to its two first-order quantifiers, with 
a universal quantifier coming first. Taking both aspects together, we speak 
of a [77^]-definition. 

Buchi’s complementation theorem says that the negation of the formula 
(*) may again be written in this form, with different Q^A^F. Biichi stated the 
result in this “logical” form (see [Bii62, Lemma 9]). Let us sketch his proof. 
Given a Biichi automaton A as above, define a congruence over A+ by 
declaring two finite words as equivalent iff the following holds: for any 
p,q ^ Q, A can reach g from p via the input u iff this is possible via the input 
V, and furthermore A can reach g fi:*om p via the input u by passing through 
a state of F iff this is possible via the input It is easy to verify that is 
a congruence with finitely many (regular) equivalence classes. Denoting the 
~^-class of the word u by [u], one observes that an a;-language [u] • [vY^ is 
either contained in L(A) or disjoint from L{A). Invoking Ramsey’s Theorem 
B ([Ra29]), one shows that any cj-word can be cut into a sequence uqUi . . . 
of finite words where all Ui for i > 1 belong to a fixed '^^-class. Applying 
this decomposition to the cj-words outside L{A)^ one sees that \ L{A) is 
representable as a union of sets [u] • [v]^, taking those pairs u, v where u • 
is not in L{A). The union is of course a finite one since there are only finitely 
many equivalence classes. This representation of A"^\L{A) is easily converted 
into a Biichi automaton. 

The complementation theorem can also be shown via a transformation of 
Biichi automata into deterministic Muller automata. This is the content of 
McNaughton’s Theorem ([McN66]). A Muller automaton is specified in the 
form A= {Q, A, qo,S, F) where S : Q x A Q is the transition function and 
JF is a subset of the powerset of Q. Acceptance of an a;-word a means that 
in the unique run p of ^ on a, the set In(p) of states visited infinitely often 
belongs to F. Formally, we express this as follows: 

(♦♦) 3p{{p{0)-qo A 'ii p{i + l) = S{p{i),a{i))) A In(/9) G ^). 

We have In(p) £ F iS for some F £ F, precisely the states in F are infinitely 
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often visited in p. A formalization of this condition reads as follows: 

V P{j) = q A /\ -. Vi 3j > i p{j) = q) 

q€F qeQ\F 

Taking the set 2^ \ ^ instead of T, one obtains an automaton recognizing 
the complement language. 

Let us analyze the logical status of deterministic Muller automata. The 
acceptance condition is a Boolean combination of iJ^-formulas, and thus we 
shall call the formula (**) a 27^ [Bool(772)]-formula. Since the run required in 
(**) is unique, one can also use a universal condition instead: 

(**') Vp (p(0) = qo ^ Vi p{i 4- 1) = S{p{i),a{i)) -> In(p) E 

This condition (*♦') is a i7i[Bool(772)] -formula, equivalent to the Z’^-defini- 
tion (**) above. Properties which are definable in 17^ -form and also in 77^- 
form are said to have a -representation. (Note that this involves two sep- 
arate definitions.) So a deterministic Muller automaton provides a zAi[Bool 
(77^)]-representation of the recognized cj-language. 

In this logical setting, McNaughton’s Theorem says that the 
definitions as provided by Biichi automata can be brought into A\ [Bool(77.2 )]- 
form, which means a “decrease” of the second-order quantifier complexity at 
the cost of a more complicated first-order kernel. 

3 Alternating automata 

The concept of alternating automaton combines the idea of existential branch- 
ing, as found in nondeterministic automata, with its dual, universal branch- 
ing. The two branching modes are specified by Boolean expressions over the 
state set Q. For example, qi V (^2 A 93) denotes the nondeterministic choice 
of going either to qi or to q 2 , qs simultaneously. The set of such positive (i.e., 
negation-free) Boolean expressions is denoted by B+(Q). We introduce alter- 
nating automata here with the so-called weak acceptance condition. It refers 
to a ranking r of the states by natural numbers. 

So an alternating automaton is presented in the form A = (Q,A,qo,S,r) 
with entries Q,A,qo as before for Biichi automata and functions S : Q x 
A B+((3) and r : Q -> {0, ...,m} for some m. Moreover, the ranking 
function r defines a stratification of Q in the sense that for any q occurring 
in an expression S{p,a) we have r{p) > r{q). This means that by applying 
transitions we can only keep or decrease the ranks of states. 

The definition of acceptance by alternating automata is somewhat in- 
volved. Usually, it refers to the notion of run tree (or computation tree). We 
use here a different terminology which allows a more elegant logical compar- 
ison with the previous modes of acceptance (nondeterministic and determin- 
istic). 
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A run of an alternating automaton is a dag (directed acyclic graph) whose 
elements are labelled with states from Q. The dag can be presented as a 
sequence of “slices” 5o,5i,..., where the states occurring in Si are the si- 
multaneously “active” ones at the i-th letter of the input word. Acceptance 
will mean that a run dag exists (which represents the existential branching 
in the automaton) such that for all paths through the dag (representing its 
universal branching) a condition regarding the ranks of the states occurring 
on this path is satisfied. 

In the definition of run dags we refer to the models of Boolean expressions 
in B+(Q). We shall identify such a Boolean model with a subset S' of Q, given 
by the assignment of states to truth values which sends the states in S to 
value 1 and the states in Q \ S to value 0. Since our expressions are positive, 
a superset S' of a model S of an expression ^ is again a model of By a 
minimal model of /3 we mean a model S of which no proper subset is again 
a model. If the expression ^ from B+(Q) is presented in disjunctive normal 
form, the minimal models of ^ are given by the sets S which constitute the 
individual conjuncts of the disjunctive form. 

Let us make precise how a run dag is built up. It is started with the slice 
So consisting solely of the initial state go (more precisely: of a node labelled 
with go)- Prom a given Si one obtains a slice Sj+i as follows, assuming that 
the input letter a{i) is the letter a: For each p from Si one chooses a minimal 
model of the expression S{p,a); the union of all these minimal models for 
p E Si is taken to be Sj+i . One inserts an edge from p G Si to q £ Si+i (more 
precisely: from the node labelled p in S* to the node labelled g in S^+i) if g 
belongs to the minimal model chosen for <5(p, a). 

If the expressions S{p, a) are given in disjunctive normal form, the forma- 
tion of a run dag can be described more easily. Again 5o consists of go only. 
Starting from a slice Si and assuming again a{i) = a, pick one disjunction 
member from S(p, a) for each p in 5i. Collect all states arising by these choices 
to form the slice 5*+!, and introduce an edge from p in Si to g in Sij^i if g 
occurs in the chosen disjunction member of S{p, a). 

Example 1. Given Q = {go, gi , q 2 ,Qs}i we may have, for a certain letter a G A: 

S{qo,a) = gi A g 2 

<5(gi , a) = (gi A qs) V (g 2 A gs) 

S{q2,a) = gi 
%3,a) = (^1 Ag2) Vg3 

With this function S, a run dag on the input word aaa . . . may start with the 
following slices: 
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Now we introduce the weak acceptance condition. This condition refers to 
the infinite paths through a run dag. By the stratification property of the 
rank function on states, the ranks on such a path will stay constant from 
some point onwards. The path is accepting if this ultimately assumed rank 
is even. Equivalently, the set Occ(r(7r)) of r-ranks occurring in the path tt 
under consideration has an even minimum. So the acceptance condition reads 
as follows: 

3 run dag p V paths tt through p : min(Occ(r(7r)) is even 

A weak alternating automaton is an alternating automaton used with this 
“weak” acceptance condition. 

In order to fix the quantifier complexity of weak acceptance, we rewrite 
the minimum condition in a more formal way. Denoting by 7r(i) the i-th state 
of the path tt, we can express it as follows: 

\J {3i r(7^(^)) = k A ^ r(7r(i)) = 1) 

k even i<k 

So weak acceptance for a given path tt is a Bool(Z'i )-condition. In the quan- 
tifier prefix “3 run dag p V paths tt” of the acceptance clause, the existential 
quantifier on run dags and the universal quantifier on paths can both be 
coded as ranging over infinite sequences over appropriate finite alphabets: 
A run dag can be represented by a sequence of slices and pointers between 
adjacent slices, so the possible entries of the sequence are representable by 
the letters of a finite alphabet. Similarly, a path through a run dag is codable 
in this way. (At this point one sees an advantage of the notion of run dag 
over the standard approach involving computation trees. Their existence is 
not directly formalizable by sequence quantifiers.) 

In summary, the definition of an cj-language by a weak alternating au- 
tomaton is a i^ 2 [Bool -representation. In comparison to the E\[Il 2 ]- 
definition given by Biichi automata, the second-order quantifier complexity 
is increased, while the acceptance condition is simpler: Instead of a require- 
ment that certain states are visited infinitely often, we only ask for the visit 
of certain states and the avoidance of others. 

In the next section we show that the class of cj-languages accepted by weak 
alternating automata is closed under complementation. As a consequence 
we can improve the i72[Bool(i7i)]-form of the definition to obtain even a 
A\ [Bool(Z'i )]-representation. 

4 Duality and weak determinacy 

The purpose of this section is to show closure under complement for weak 
alternating automata. In this analysis, game-theoretic notions are useful. 
With each (weak) alternating automaton A = (Q,A,qo,S,r) and each u- 
word a € we associate an infinite game G{A,a), played by two persons 
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called Automaton and Pathfinder. The idea is that in the process of scan- 
ning the input word a, Automaton picks sets of simultaneously active states 
according to the transition function of A, whereas Pathfinder picks, at each 
point, one of these momentary active states. Making such choices in alterna- 
tion they build up a path through a run dag of ^ on a, and Automaton is 
declared the winner of the play if the acceptance condition of A is satisfied 
on this path. 

Formally, a game position refers to the number i supplying the momentary 
input letter a{i). If Automaton has the next move, the game position is of 
the form (i,p) with p and if Pathfinder has to make the next move, the 
game position has the form (i,5) with S CQ. The initial position is (0,^o); 
so Automaton starts. A move of Automaton from position (i,p) consists 
in the choice of a minimal model of S{p,a{i)), i.e. a set 5 C Q, yielding 
the game position (z + 1,5). Pathfinder reacts by picking a state s from 5, 
producing the game position {i -f- 1,5). The play determines a sequence of 
states (extracted from the positions of Automaton), and Automaton wins 
the play if the minimal rank occurring in this sequence of states is even. 

A local strategy (also called memoryless strategy) for a player P is a func- 
tion which associates with any game position of P a move which can be 
performed in this position. Such a function is called a winning strategy for 
player P from game position pos if its application will produce, when starting 
from po5, for any moves of the opponent, a play won by P. 

Proposition 1. The weak alternating automaton A accepts a iff in the game 
G{A^a), Automaton has a local winning strategy from the initial position. 

Proof. First assume that there is an accepting run dag of ^ on a, say with 

slices 5o,5i, Define a strategy for Automaton by choosing, given game 

position (z,p), the set 5 of states from Si.^i which are reachable from p by an 
edge of the run dag. In this way, starting from position (0,qo), Automaton 
ensures that the play proceeds along a path through the run dag. Since the 
run dag is accepting. Automaton wins by this local strategy. 

Conversely, a local strategy for Automaton defines an accepting run dag: 
For i = 0,1,... the slices Si are built up inductively, beginning with the 
singleton So = {g'o}: For any game position (i,p) as picked by Pathfinder, 
Automaton’s local strategy prescribes a set of states as next move; the union 
of these is taken to form 5i+i , and edges are inserted which allow to trace 
the connections to the different choices of p. It is clear that the constructed 
run dag is accepting. □ 

The complementation proof for weak alternating automata will be given 
in this game-theoretic setting. It has two steps: the dualization of alternating 
automata and a determinacy result for infinite games. 

For the first step, we introduce the dual automaton A oi 8i given weak al- 
ternating automaton A = (Q,A,qo,6,r). The definition uses the dualization 
of Boolean expressions: Given an expression /3 G B+(Q), let its dual ^ arise 
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from /3 by exchanging V and A. Now the dualized transition function 6 is de- 
fined by J(p, a) = S{p, a). The dual automaton A is obtained as (Q, A, go? 
with the convention that in an accepting run dag, on each path the minimal 
rank of visited states should be odd. 

We need a remark on models of the dual of an expression in (Q). Recall 
that a model of an expression 13 is considered as a subset S of Q. 

Remark 1. A set 5 is a model of 0 iff every minimal model R of /3 contains 
a state from S. 

Proof. Let MM{/3) be the set of minimal models of (3. We have the logical 
equivalence 

V 

ReMM{p)qeR 

By duality, we have 

A 

ReMM{(3)q€R 

This shows the claim. □ 

Now we are able to connect local winning strategies in the two games 
G{A,a) and G(-4,a): 

Proposition 2. Automaton has a local winning strategy in G(A,a) from the 
initial position iff Pathfinder has a local winning strategy in G{A, a) from the 
initial position. 

Proof. We show how to transform a local winning strategy of Automaton 
in G{A,a) into a local Pathfinder strategy for the dual game. The desired 
strategy in G{A,a) has to tell Pathfinder which state to take for any game 
position (i -h 1,5) (where i > 0). Note that in fixing the strategy it suffices 
to consider only game positions (z H- 1, 5) which are reachable, i.e. for which 
a sequence of moves in G{A,a) exists starting in the initial position (0,go) 
and ending in (z -f 1, 5). The set 5 of the game position (z 4- 1, 5) is produced 
by Automaton from a game position (z, s), such that 5 is a minimal model of 

S{s,a{i)). Pathfinder chooses such a state s which could produce 5 via a(z). 
Now in the game G{A^a) at position (z,5), the given local winning strategy 
of Automaton picks a minimal model R of S{s,a{i)). By the remark above, 
there is a state in i? D 5. For his move from the game position (z -f- 1,5), 
Pathfinder chooses such a state. Then in G(A, a) a state sequence is built up 
which is compatible with Automaton’s winning strategy in G{A^ o) and hence 
is won in that game by Automaton. In the game G{A, a), where the roles of 
even and odd ranks are exchanged, this state sequence gives a play won by 
Pathfinder. So the described strategy is a winning strategy for Pathfinder in 
G{A,a), and its specification shows that it is local. 

The other direction is shown analogously, by exchanging the roles of A 
and A. □ 
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Proposition 3. Let A be a weak alternating automaton. From any game po- 
sition in G{A,a)j either Automaton or Pathfinder has a local winning strat- 
egy- 

Proof. Let A = {Q,A,qo,S,r) be a weak alternating automaton, where r : 
Q {0, . . . ,m} and Qi {q £ Q \ r{q) = i}. Let Posq be the set of game 
positions of the game G{A,a). It is divided into the sets Posa and Posp 
where Automaton, respectively Pathfinder has the next move. 

As a preparation we need the definition of “attractor set of a set T of 
game positions”. This attractor set (for player Automaton, say) is denoted 
Attr>i(T); it contains all game positions from which Automaton can force 
in finitely many moves a visit in the target set T. The set is constructed 
inductively by collecting, for i > 0, the positions from where a visit to T can 
be forced within i moves: Let Attr^ := T and set 

Attr^+i(T) := Attr‘;i(T) 

U {p G Posa \ there is a move fi:om p to Attr^(T)} 

U {p G Posp I all moves from p lead to Attr^(T)} 

Now let Attr a{T) = Attr^(T). Prom the positions in this set player 
Automaton can force a decrease of distance to T in each step (which defines 
a local strategy). Also note that for the game positions pos outside Attr>i(T), 
Pathfinder will be able to avoid entering this set. (If at pos it is Pathfinder’s 
turn, one move to the complement of Attr^(T) is possible, and if it is Au- 
tomaton’s turn, all moves lead to the complement of Attr^(T); otherwise pos 
would be already in Attr>i(T) itself.) So from outside Attr^(T), Pathfinder 
can avoid, by a local strategy, to enter this set and hence can avoid the visit 
of T. 

The set Attrp(T) for player Pathfinder is constructed analogously. 

Using the notion of attractor set and corresponding local strategies we 
determine inductively the game positions in G(>1, a) from where player Au- 
tomaton, respectively Pathfinder, wins. 

Clearly, from the positions in Aq := Attr>i((3o)j Automaton can force, 
by a local strategy, to reach states of rank 0 and thus win. Consider the 
subgame whose set of positions is Pos\ := Posq \ Aq (all of which have rank 
> 1). Prom the positions in A\ \= kttxp{Pos\ fl Qi), Pathfinder can force, 
again by a local strategy, to reach (and stay in) states of rank 1 and hence 
win. (Note that Pathfinder can avoid to enter Aq, as explained above.) In this 
way we continue: In the game with position set P 0 S 2 := Posi\A\ (containing 
only states of rank > 2) we form the attractor set A 2 := Attr^(Po52 H Q 2 ), 
etc. Then the positions from which Automaton wins (by the local attractor 
strategies) are those in the sets A{ with even i < m. Similarly, Pathfinder 
wins from the positions in the sets Ai with odd i < m (again by his local 
attractor strategies). □ 

Now we have all prerequisites for the complementation of weak alternating 
automata: 
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Theorem 1. For any wmk alternating automaton A over the alphabet A, 
we have \ L{A) = L{A). 

Proof. By Proposition 2, the automaton A does not accept the input word 
a iff Automaton does not have a local winning strategy in G(A, a) from the 
initial position. By Proposition^, this means that Pathfinder does not have 
a local winning strategy in G{A, a) from the initial position. By the deter- 
minacy result (Proposition 5), this holds iff Automaton has a local winning 
strategy in G{A., a) from the initial position, which in turn means that A 
accepts a. □ 

The present game-theoretic complementation proof for Biichi automata has 
the same general structure as the complementation of nondeterministic tree 
automata in the framework of parity games (or “Rabin chain games”; see 
for example [Th97]). So it is possible to compare the two proofs. (Note that 
the two classical proofs of Biichi automata complementation, via determin- 
istic automata or via a finite congruence saturating the given a;-language, 
do not extend - as far as we know - to tree automata, which makes a di- 
rect comparison difficult.) The parity games associated with tree automata 
have a winning condition defined by a Boolean combination of 
and the corresponding determinacy proof, also by induction on the ranks 
of game positions, involves a nontrivial combination of attractor strategies 
with strategies given by the inductive hypothesis. In the “weak” games con- 
sidered in the present paper, where Boolean combinations of -conditions 
serve as winning conditions, a straightforward reachability analysis, yielding 
the attractor sets Aq, . . . , Am^ suffices for the determinacy proof. This direct 
comparison in the game-theoretical setting shows in which sense complemen- 
tation is easier for cj-automata than for tree automata and thus reveals a 
characteristic difference between o;-automata theory and tree automata the- 
ory. 



5 Equivalence of Biichi automata and weak alternating 
automata 

In order to complete the complementation proof for Buchi automata, we 
have to supply transformations from Biichi automata to weak alternating 
automata and conversely. These transformations, as developed by C. L5ding 
in [L698] , are described here very briefly. 

Proposition 4. For any Biichi automaton A there is a weak alternating 
automaton A' with L{A) = L{A'). 

Proof Let A = (Q^A^qo^A^F) be a Biichi automaton with n states. The 
desired weak alternating automaton is constructed over the state set Q x 
{0, ...,2n}, taking (qo,2n) as initial state, and defining the rank function 
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r by r((g,i)) = i. The transition function S associates to a state (p, 0) of 
rank 0 and letter a simply the disjunction over all (g, 0) with (p,a,q) G A. 
For even ranks i > 0, however, we take the disjunction over all expressions 
{q, i) A{q,i — 1) with (p, a, q) £ A. This will open a track of states of odd rank 
until some final state of the Biichi automaton is reached: Namely, as long as 
p ^ F we set, for odd i, (5((p, z),a) to be the disjunction over all (q,i), again 
of rank z, with (p, a, q) E A, while for p G F we take the disjunction over all 
(g, i — 1) with (p, a,q) G A. 

From an accepting run p of A one obtains an accepting run dag p' of 
A' and conversely. The construction of p' from p is straightforward: With 
the states of even rank, one simulates the given run p, and by the definition 
of the transition function 5 of ^ no path eventually stays on an odd level. 
The converse direction requires to compose an accepting Biichi run p from 
an accepting run dag p'. This composition is achieved by concatenating run 
segments leading from a state (p,2z) to a state (q,2j) with i > j, for in this 
case an intermediate visit in F is ensured by the construction of 6. In order 
to be able to do this infinitely often, one has to reset the (even) rank to a 
higher level infinitely often. This in turn is made possible by the presence of 
n -f 1 even ranks (from 0 to 2n), which means that once rank 0 is present, 
some state occurs on two ranks. □ 

Proposition 5. For any weak alternating automaton A, there is a Biichi 
automaton A^ with L{A) = L{A'). 

Proof. Apply a subset construction as given by Miyano and Hayashi [MH84] : 
The desired Biichi automaton A' has states (5, R) where 5, R are subsets 
of the state set Q of the given weak alternating automaton A. In the first 
component 5, A' guesses the slices of a run dag of A and thus a run dag, 
while in the second component R keeps track of those states from S which 
are of odd rank. If they eventually vanish, R is reset to the whole set S again. 
Then infinitely many such reset operations (captured by the Biichi acceptance 
condition) signal that no path in the run dag finally stays in states of odd 
rank. □ 



6 Discussion 

We have outlined a complementation proof for Biichi automata by invoking 
a determinacy theorem on weak infinite games. Referring to the logical clas- 
sification of automata theoretic definability explained in Section 2, we passed 
from 27^ [Tl^J-representations of (j-languages, as given by Biichi automata, 
to the level of zi2[Bool(I7°)]-representations, as given by weak alternating 
automata. This is an alternative to the option to pass to Zli[Bool(i72)]- 
representations, as provided by deterministic automata. 

None of the steps as described in the propositions above is very difficult; 
so the complementation proof via weak alternation is composed of simple 
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“modules”, in some contrast to the two classical proofs (which rely on a 
nontrivial combinatorial result or a complicated automaton construction). 
Of course, there is a price to be paid in using the more involved definition of 
acceptance of alternating automata. 

Another conceptual advantage of weak alternating automata may be the 
fact that acceptance is defined without resorting to liveness conditions; in the 
kernel of the second-order definition of acceptance one finds here only con- 
ditions on mere reachability or non-reachability of states. This phenomenon 
may be helpful in the investigation of still unsolved problems of a;-automata 
theory, for instance in the question of finding a good framework for the min- 
imization of Lj-automata. 
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Summary. We study the family of languages accepted by the integer weighted 
finite automata. Especially the closure properties of this family are investigated. 



1 Introduction 

The integer weighted automata, as were studied in [4], [5] and [6], are closely 
related to the 1-turn one-counter automata as considered by Baker and Book 
[1], Greibach [3], and especially by Ibarra [7]. In our model the counter is 
replaced by a weight function of the transitions, and while doing so, the 
finite automaton becomes independent of the counter. The differences be- 
tween one-counter automata and integer weighted finite automata as well as 
the deterministic integer weighted automata are considered in [4]. 

We shall first give the definition of the weighted finite automata. 

Consider a (nondeterministic) finite automaton A = (Q,A,S,qo) without 
final states with the states Q, the alphabet A, the set of transitions S C 
Q X A X Q and the initial state qq. 

We shall redefine the transitions using a set of edges T = {^1,^25 ■ • • ? ^m} 
and a transition function a:T S. We do this to allow transitions U and tj^ 
where i ^ j, but a{U) = cr{tj). In other words, there may exists many copies 
of one transition in this new set of edges. Clearly this new definition of the 
transitions does not affect the language accepted by the automaton. 

Let G, or be a group with identity l. A (G-) weighted finite au- 

tomaton A^ consists of a finite automaton A = ((3,A, o', go) a^s above, and 
a weight function 7: {ti, . . . ,tn} — > G of the edges. To simplify the notation, 
we shall write the edges in the form 

t= {q,a,p,z) 

if (r{t) = (g, a,p) and 7(t) = Similarly, we shall write the transition function 
cr as a set, crCQxAxQxZ, where 

cr = {{q,a,p,z) \ 3t ^T\(r{t) = (q,a,p) and jit) = z} . 

In the figures we shall denote such an edge t by 
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But note that the function 7 does not affect the computations of the finite 
automaton A. It will be used only in the acceptance of words. 

Let 7T = - --tin be a path of A, where cr(tij) = for 

0 ^ j < n — 1. Define a morphism || • ||: {ti, . . . by setting ||t|| = a 

if cr(t) = (q,a,p). The weight of the path tt is the element 



7(7 t) = 7(«io)7(*ii) • ••liUn) G G. 



Further, we let L{A^) = II7 ^(t)ll> *^hat is, 

LiA^) = {w ^ A* \ w = ||7 t||, 7 (7 t) = i}, 

be the language accepted by A^. 

A configuration of A'^ is any triple (q,w,g) e Q x A* x G. 

A configuration (q,aw,gi) is said to yield in a configuration (p,w,gig2), 
denoted by 

(q,aw,gi) [=a^ {p,w,gig2), 

if there is an edge t such that cr{t) = (g, a,p) with j{t) = g2- Let or 

simply \=* if A^ is clear from the context, be the reflexive and transitive 
closure of the relation [=^7 . 

We shall restrict to the case, where the group of the automaton is the 
additive group of integers, namely (Z, -f-, 0). Such automata are called integer 
weighted finite automata and denoted by FA{Z). 

Note that our definition of the integer weighted finite automata is a re- 
stricted case of the extended finite automata of Mitrana and Stiebe [8]. In 
the extended finite automata the underlying automata has final states F CQ 
and the transitions reading the empty word are allowed. 

Let be an FA{Z). The empty word is always included in L{A^), since 
in an integer weighted finite automaton all states, including the initial state, 
are final. Therefore we known that not all regular languages can be accepted 
by an FA{Z). On the other hand. 

Theorem 1. For each regular language L, there exists an FA{Z) A^ such 
that L{A'^) = L U {e}. 

Proof. Let L be a regular language and let A = (Q, A, <5, go 5^) be a (non- 
deterministic) finite automaton such that L{A) = L, where 6 C Q x A x Q. 
We may assume that A has one initial state go and one final state g/ (or two 
final states go and g/, if e: E L) such that there are no transitions to go and 
no transitions from g/. It is well known that such an A exists for all regular 
languages L. 

We shall define an FA{Z) A'^ = (Q, A, cr, go), where cr is a bijection and 
therefore we may define T = S. One such required weight function 7 is defined. 
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for (p, a, q) £ S, by 



7(P, a, q ) 



^0 if p = qq and q - qf, 
1 iip = qo a.nd q:^qf, 
-1 if go and g = gf/, 

,0 if p go and g ^ qf. 



( 1 ) 



It is obvious that 7(7 t) = 0 only in the case where tt is an accepting path 
of A. This proves our claim. 



As we mentioned, not all regular languages are accepted by a FA{Z). On 
the other hand, it is easy to show that not all languages accepted by integer 
weighted finite automata are regular, since the language 

L = {a^b^ I n > 0} 

is accepted by an FA{Z) of Figure 1, but L is not regular. 



(a, 1) 



Qo 



(^- 1 ) 







Fig. 1. An FA{Z) accepting the language {a^b^ I ^ > 0}. 



It was proved in [5] that the universe problem, asking whether all input 
words are accepted by an integer weighted automata, is undecidable. Actually, 
it was proved there that the universe problem is undecidable for a rather 
restricted type of 4-state integer weighted finite automata. In this type the 
edges on every possible path have first positive weights, then zero weights, 
then negative weights and then again zero weights. Any of these parts may be 
trivial. Such a restricted type of integer weighted finite automaton is called 
unimodal 

2 Closure properties 

In this section we consider the closure properties of the family of FA{Z) 
languages, denoted by Cfa{Z)^ under various operations. We shall begin with 
the union. 

Theorem 2. The family of F A{7 j) languages is closed under unions. 
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Proof. Let Li and L2 be two FA{Z) languages accepted by the FA(Z)’s , 
where A = {Qi, A,ai,qi), and where B = (Q2, ct 2, 92), respectively. 
Assume that Qi f 1 Q2 = 0 - We define an FA{Z) where 

C = (Qi UQ ‘2 U {qo} ,A,a,qo), 

and qo ^ Qi U Q2 is the new initial state. The edges in are as in A^ and 
B", except that there is an edge {qo,a,p, z) for p E Qi U Q2 in if the edge 
(^i,a,p,2;}, where p E Qi, is in A^ or the edge {q:i,a,p,z), where p E Qi, is 
in B. Clearly L(C^) = l(A^) U L{B^) = Li U Li. This proves the claim. 

Next we consider the intersection. For this, let Li = a’^ I n > 0 } and 
L2 = {a^b^ I n > 0} c* . These languages can be accepted by integer weighted 
finite automata, but 



L = LiC\L 2 = {a^b^c^ I n > 0} 

cannot be, since L is not even a context-free language and it was proved in [ 4 ] 
that Cfa{z) C Ccf, where Ccf denotes the family of context-free languages. 

Theorem 3. The family of FA{Z) languages is not closed under intersec- 
tions. 

Next we consider the concatenation operation on languages. 

Theorem 4. The family of FA{Z) languages is not closed under concate- 
nations. In fact, the square of a language L E Cfa{Z) need not be in 
^FA{Z)- 

Proof. Let L = {a^b^ I ^ > 0 }- We shall show that the (one-counter) lan- 
guage 

= L L = {a^b^a^b^ | n, m > 0} 
cannot be accepted by an FA{Z). 

Assume on the contrary that there is an FA{Z) A'^ with states Q and 
weight function 7 such that it accepts L‘^. Since L‘^ is an infinite language 
and it contains words that have length greater than |Q|, necessarily there is 
a cycle in A. We have now two cases to consider: 

1) Suppose A has only one cycle. Assume that the weight of the cycle is 
d. To accept all words in L^ with only one cycle, necessarily d = 0 , since L^ 
is infinite. Let uv £ L‘^,u,v ^ L and \uv\ > |Q|. Then we have a factorization 
uv = xyz, where y is read during the cycle. Since uv = xyz gives a path of 
weight 0 , we get that xy^z E L{A'^) for all k £ N. But L^ does not contain 
any words xy^z for A: > 2 and y ^ e. 

2 ) Suppose that A'^ has more than one cycle. If there exists a cycle of 
weight zero in any accepting path, then we get a contradiction as in the 
previous case. On the other hand, the weights of all cycles cannot be of the 
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same sign, since is infinite. It follows that there must be E L such 
that uv = xyzrs, and, in an accepting path tt = 'Kx'^y'^z'^r'^s of i/v, y is read 
during the cycle TTy and r is read during the cycle tt^, and 7(7Ty) and j{7Tr) 
are of different sign. But now the word 

Xykbi^r.)\ + l^^k\^M\ + lg g 

for each A: > 0 is accepted by the path 



Since 

7(7t') = 7(7T*) + 7K)(A:|7{7rr)| + 1) + i{-Kz) + lM{k\7(-Ky)\ + 1) + j{ns) 
= 7(7Tx) + 7(7ry) + + ^{Kr) + 

+/j(7(7rj,)|7(7rr)| + 7(Trr)|7K)|) = 0. 

Since these new words are not in we get a contradiction. 

Note that although the family of FA{Z) languages is not closed under in- 
tersections and concatenation, it is closed under these operations with regular 
languages (that contain the empty word). This is stated in the next theorem. 

Theorem 5. Let R he a regular language with e G R and L be an FA{Z) 
language. Then L, RL and LR are FA{Z) languages. 

Proof. Assume that R is accepted by the FA B = {Q, A, S, go? F) and L with 
the FA{Z) where A = (P,A,ai,po). Assume also that Q H P = 0. We 
can transform B into a B^ accepting R, where a is defined as in 1. We may 
also assume that in A'^ all edges have even weight, since we can multiply each 
weight by 2, and the accepted language remains the same. 

For Pn L, we define where C = {Q x P, A,cr, (qo,Po)), and there is an 
edge 

{{Q,p),a, {r,s),{zi + Z2)) 

in C^, for all edges {q,a,r,z\) in 5“ and (p,a,s,Z 2 ) in . By the definition 
in 1 of a, the weight of a path starting from (qo,po) and ending in a (q,p) 
is 0 (mod 2) if and only if g £ F, which is equivalent to the fact that the 
word is in R. It follows that ii; E A* is in i? fl L if and only if it is in L{C^). 

For RL, we construct an FA{Z) where C = {Q \J P,A,a,qo), by 
connecting the two FA{Zys by introducing new edges 



(/, a,p, z), 

for / E F and a E A, if ^ 2 ;} is an edge in A^ . It is then clear that 

L(C^) = RL. 

For LR, the same construction can be used, but this time we connect the 
two FA{Z) in the opposite order. 
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Next we shall consider the star operation. Let be a language and 

oo 

K* = U K\ 

i=0 

By using again the language L = I ^ > 0} and the proof of Theo- 

rem 4, we can show 

Theorem 6. The family of FA{Ij) languages is not closed under star. 

Proof. Assume that there is an FA{Ij) accepting L* for L = {a^b'^ | n > 
0}. It follows by Theorem 5 that the language 

= L^ 

can be accepted by an FA(Z), since a*b*a*b* is a regular language. But this 
is a contradiction by the proof of Theorem 4. 

The family of FA{Z) languages is not closed under complement, since 
each FA(Z) language contains the empty word, and therefore it is not in the 
complement. But let us consider the complement modulo e. For any language 
L C A* y the complement modulo e of L is 

L, = (A*\L)U{e}. 

For example, let A = {a, 6} and L = {a'^b'^ I ^ > 0}. Now there is a 
partition 



= a* U bA* U aA*baA* 

U {a^b'^ 1 0 < m < n} U {a^b^ | m > n > 0} , 

and this language can be accepted by the F A(1j) in Figure 2. In other words, 
to prove that the family of FA(Z) languages is not closed under complement 
modulo e:, we have to use some other language than L. It is also clear that 
this new language cannot be regular, since the family of regular languages is 
closed under complement. 

Let A = {a,6,c} and S = {a^b^c^ 1 n > 0}. Now S is not an FA{Z) 
language, since it is not a context-free language. But 

5e = a* U {b, c} A* U A^baA* U A*c {a, b} A* 

U \ n ^ k, A:,m, n € N} 

U {a^b'^c^ I n 7^ m. A:, m, n G N} . 

is a FA{Z) language, since the languages on the first row are regular and the 
last two languages are easily seen to be FA{Z) languages. Since 

W)e=S, 



we may write 
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(«,0) (6,0) (a,0) (6,0) 




Fig. 2. An FA{Z) accepting the complement of {a”6^ I ^ ^ 0} modulo e. 



Theorem 7. The family of FA{Z) languages is not closed under complement 
modulo e. 

It is obvious that the family of FA{Z) languages is closed under taking the 
image of a nonerasing morphism h : A* J5*, since each transition reading 
a letter a E A can be replaced by a new path, which reads the image h{a) 
and has the same weight as the original transition. Note that the morphism 
must be nonerasing, since otherwise we would get e-transitions. 

Next we consider inverse morphisms. 

Lemma 1. Let h : B~^ -> A+ be a morphism and where A = (Q, A, a, qo), 
be an FA{Z). Then there exists an FA{Z) such that 

L{B^) = = {h~\w) I w e LiA’)} . 

Proof. Let where B = {Q,B,cr' ,qo), and the edges of B°“, for ail q € Q 
and b G B, are defined by 

(q,b,p,z) if {q,h{b),0) \=A-< (p,£,z), 
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where p £ Q and z £ Zi. Note that if h{b) = e, then there is a loop 



{Q,b,q,0), 

for all q £ Q. Now it is straightforward to show that {qo,v, 0) {q, e, 0) if 

and only if (qo,h{v),0) \=A"r (q,£,0). This proves the claim. 

We have proved 

Theorem 8. The family of FA{Z) languages is closed under taking the im- 
ages of nonerasing morphisms and under taking the image of arbitrary inverse 
morphisms. 

Next we shall consider the shuffle operation, denoted by lu. For u,v £ A* 
and a, 6 E A, the shuffle is defined recursively by 

1. {au in bv) = a{u\iibv) Ub{auin v), and 

2. (e LU w) = (u HI e) = {w}. 

For two languages K,L C A*, define the shuffle by 

KujL= (mnv). 

v^Li 

Theorem 9. The family of F A(Z) languages is not closed under shuffle. 

Proof. Let L\ = {a^b^ I ^ > 0} ^nd Lz = {c^d^ | m > 0}. We shall prove 
that Li LU L ‘2 is not an FA{Z) language. 

Assume on the contrary that L\ LU L 2 is an F A{Z) language. Then by 
Theorem 5 the language 

(Li LU L2) n {a'^b'^c^d'^ U {e}) = Li • L 2 

is also an FA{Z) language, since a'^b^c'^d^ is regular. Since L\ • Li has a 
morphic image Lf, also L\ is an FA{Z) language. But this contradicts the 
proof of Theorem 4. 

Note that all these closure properties hold also for unimodal FA{Z) lan- 
guages, except that in the case of inverse image of a morphism we have to 
assume that the morphism is nonerasing. Closure properties for unimodal 
and deterministic integer weighted finite automata are studied in [4] . 

3 Final states 

In this section we shall consider the integer weighted finite automata with 
final states. This is done for the purposes of further closure properties of 
FA{Z) languages. 
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An integer weighted finite automata with final states is defined as the 
integer weighted finite automata in Section 1 except that the underlying 
automata has final states, i.e. the underlying automata is ^ = (Q, A, S, 
where F C Q is the set of final states. Let a be an edge set and 7 be a weight 
function for A. The accepting of is now defined by the final states, in 
other words, we let 

LiA'^) = {w e A* \ (qo,w,0) \=a^ (^,^,0) for some q G F}, 



be the language accepted by A^ . 

We shall denote the family of languages accepted by integer weighted 
finite automata with final states by 



Theorem 10. ~ ^FA{Z) U 

Proof It is obvious that Cfa(z) Q ^FA(zy FA{Z) all states are 

final. We need to show that each language L G ^fa(z) accepted by 

an FA{Z) or LU e: is accepted by an FA{Z). 

Consider now an arbitrary integer weighted finite automaton A'^ with final 
states, where A = (Q,A,a,qo,F). We have two cases depending on whether 
or not £ is in L{A'^): 

(i) Assume that e G L{A^). This implies that qo G F. The only thing 
we shall change in this case is the weight function 7. For (q,a,p,z) G cr, let 
(q,a,p,z*) be in ex', where 



' 2z 1 if q = qo and p ^ F, 

2z if p,q ^ F or p,q G F, 
2z 1 if q G F and p ^ F, 
<2z — 1 if q ^ F and p G F. 



We define an FA{Z) B", where B = (Q,A,a',qo). By the definition of the 
edges in the weight of a path is 0 (mod 2) only when we end in a state 
of F. If we axe in the state in F then the weight of the path is 2w, where 
w G Zis the weight of the same path in A'^. This proves that L{B^) = L{A'^). 

(ii) Assume now that e ^ L{A"^). Since qo is not a final state, we must 
assure that the only word accepted in the initial state is e. Therefore we define 
a new initial state q'o ^ Q- The edges from ^re defined in the following 
way. Let 

a' -al}{{q'Q,a,p,z) \ {qo,a,p,z) e a). 

Now the automaton B {qo} ,A.,a' .(f^.F 1} {qQ}) accepts the language 

L{A^) U {£:}. The claim follows then by the case (i). 



We proved that each nonempty language in C^fa{Z) accepted with 

integer weighted finite automaton where all states are final or all but the 
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initial state are final. In an integer weighted finite automata with final states 
the number of final states does not make any difference, since it can be proved 
that each language in accepted by an integer weighted finite 

automaton with zero, one or two final states depending on whether e is in 
the language or not. This follows, since we may define a new final state and 
make a copy of each edge ending in the final state to this new final state. No 
outgoing edges are defined for this new state. Clearly all accepted nonempty 
words are accepted also in this new final state. To have e accepted we must 
also have the initial state as a final state. Naturally, the empty language can 
be accepted only by zero final states. 

It is obvious that the closure properties proved for the family Cfa(z) in 
Section 2 also hold for the family difference that Theorem 5 

can be stated as follows: 

Theorem 11 . Let R be a regular language and L be in LpA{z)‘ 

RL and LR are CpA(Z) 

The reason for studying the integer weighted finite automata with final 
states is that, by Theorem 11, we get that LpA{Z) ^ semi-AFL^ where AFL 

stands for abstract family of languages. Indeed, LpA(z) meaning that 

it is closed under e-free morphisms and inverse morphisms, and intersections 
with regular languages. Furthermore, a trio is a semi-AFL, if it is closed under 
imion. For these definitions and for properties of a semi-AFL, we refer to [2]. 

A family of languages is called a AFL if it is a semi-AFL and closed under 
concatenation and Kleene plus. Obviously CpA(z) AFL, since by 

Theorem 4 it is not closed under concatenation. Neither can it be closed under 
Kleene plus, since by Theorem 3.1.2 in [2] this would imply that CpA{Z) 
an AFL. This also follows by the proof of Theorem 6 . 

A finite transducer with accepting states is a 6 -tuple M = {K,A,B, 
H,po,F), where AT is a finite set of states, A is the input alphabet and 
B is the output alphabet, H is finite subset of AT x A* x B* x K, po £ K 
is the initial state and F C A" is the set of final states. The elements in H 
are called moves. A finite transducer is a finite automaton with output, i.e. 
a move (p, a, 6 , q) means that when we are in the state p and read a E A* as 
input, we move to the state q and output b £ B* . 

As usual, we define the relation j=x on K x A* x B* by (p,xw,z) \=m 
( q,w,zy) for each w £ A* if {p,x^y,q) is in H. The triple (p,w^z) represents 
the fact that M is in state p, w is the input still to be read and z is the 
output so far. 

For each word w £ A*, we define the set 

M{w) = {z£B" \ (po,w,e) (q,e,z), and 9 G F}, 

where denotes the reflexive and transitive closure of \=^m- The mapping 
M from 2 ^* into 2^* is called a rational transduction. 
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Note that M is £:-free if M(w) is e-free for all w ^ e and M(e) contains 
e if and only if po ^ 

In [2], Corollary 2 of Theorem 3.2.1 states that each trio is closed under 
e-free rational transduction, and therefore 

Theorem 12. The family under e-free rational transduc- 

tions j i.e. for all L G ^fa(Z) s-free finite transducers M 

M{L) e CpA(zy 

A finite transducer Q = (K^A^B^H,po,F) is called a generalized se- 
quential machine {gsm for short), if H CKxAxB*xK.G is e-free, 
if (p, a, e,q) ^ H for all p, g G AT and a G A. The gsm mapping is defined 
as the rational transduction, and the inverse of a gsm mapping is called an 
inverse gsm mapping. 

By Corollary 3 of Theorem 3.2.2 in [2], each trio is closed under inverse 
gsm mappings. Therefore 

Theorem 13. The family Cpj^^x) dosed under e-free gsm mappings and 
arbitrary inverse gsm mappings. 

The closure under e-free gsm mappings follows from Theorem 12. 

Let A and B be alphabets. A function r: A* 2^* is called a substitution.^ 
if 

1. r(e) = e, and 

2. r{xy) = r(x)r{y) for all x, 2 / G A*. 

A substitution r is extended to languages by defining r(L) = L}yj^LT{w). A 
substitution is said e-free, if r(a) is e-free. 

By Theorem 3.3.1 in [2], we have the next theorem. 

Theorem 14. Let r: A* -> 2^"^ be a substitution such that, for all a £ A, 
r{a) is an e-free regular language, and L G CpA- Then r(L) G 

Finally, we give a negative result on the closure under e-free substitution. 
By Proposition 3.3.3 in [2], each trio closed under e-free substitutions is an 
AFL. 

Theorem 15. The family Cpj^ is not closed under e-free substitutions. 
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A Power Series Approach 
to Bounded Languages 



Juha Honkala 



Summary. We use the framework of the monograph ” Semirings, Automata, Lan- 
guages” by Kuich and Salomaa to study bounded languages and algebraic series 
having bounded supports. We characterize A-aJgebraic series having bounded sup- 
ports in the case A is a subring of the real numbers or a positive semiring. This 
result implies the wellknown characterization of bounded context-free languages 
and shows that the characterization holds true for languages of finite degree of 
ambiguity even if multiplicities are counted. 



1 Introduction 

Classical language theory can be viewed as a study of formal power series over 
the Boolean semiring. In their monograph ’’Semirings, Automata, Languages” 
Kuich and Salomaa give a presentation of automata and language theory over 
an arbitrary semiring. Kuich and Salomaa also use formal power series as an 
efficient tool to obtain deep decidability results (see also Salomaa [21]). 

In this paper we use the framework of Kuich and Salomaa to study 
bounded languages and algebraic power series with bounded supports. We 
first show that for a commutative semiring A the A-algebraic series having 
bounded supports form a semiring and characterize this semiring in the case 
A is a positive semiring or a subring of the real numbers. This result is used to 
give a simple proof of the characterization of bounded context-free languages 
due to Ginsburg and Spanier [5]. It is also deduced that the result of Gins- 
burg and Spanier holds true for languages having finite degree of ambiguity 
even if multiplicities are taken into account. 

For further background and motivation see Kuich and Salomaa [15], Ro- 
zenberg and Salomaa [20] and Ginsburg [4]. Also all unexplained terminology 
and notation is from these references. 

2 Definitions and earlier results 

Suppose A is a commutative semiring and X is an alphabet. The set of 
formal power series (resp. polynomials) with noncommuting variables in X 
and coefficients in A is denoted by A <C X’*' > (resp. A < X* >). The set 
of A-algebraic (resp. A-rational) series with noncommuting variables in X 
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is denoted by X* ^ (resp. X* ':$>). If w G X*, the series 

r £ A ^ X* ':$> is called w-unary if 

supp(r) C T/;*. 

A series rGA^CX^^^is called unary if there is a word w £ X’^ such that 
r is ly-unary. 

In this paper we discuss A-algebraic series having bounded supports. By 
definition, a series r £ A X* ^ is s-bounded if supp(r) is a bounded set, 
i.e., there exist an integer m > 1 and nonempty words it;i, . . . , Wm £ X* such 
that 

supp(r) Cwlw 2 ...w*^. 

Clearly, unary series are s-bounded. 

Lemma 1. Suppose A is a commutative semiring. //ri,r 2 £ A <C X* 
are s-hounded, so are ri H- r 2 and rir 2 . Consequently, the set of s-hounded 
A-algebraic series is a semiring. 

Proof. Suppose ri and r 2 are s-bounded. Then there exist integers m,n > 1 
and nonempty words Ui,vj £ X* , 1 < i < m, 1 < j < n, such that 

supp(n) Culu* 2 ...u*^ 



and 

supp(r 2 ) C V1V2 . . 



Therefore 

supp(ri -f T 2 ) C supp(ri) U supp(r 2 ) C u{u2 . ■ • Um'^lv2 • • • 



and 

supp(rir 2 ) C supp(ri) • supp(r 2 ) C ulu^ . ..u*^vlvl . . 

Hence r\ -h r^ and rir 2 are s-bounded, and the set of s-bounded A-algebraic 
series is a subsemiring of A X* >. □ 

Let B = {0, 1} be the Boolean semiring. The semiring of s-bounded B- 
algebraic series is isomorphic to the semiring of bounded context-free lan- 
guages. This semiring plays an important role in classical language theory, 
see Ginsburg [4]. It is also the natural framework to study thin and slender 
languages introduced in Andra§iu, Dassow, Paun and Salomaa [1]. (For thin 
and slender languages see also Dassow, Paun and Salomaa [2], Paun and Sa- 
lomaa [17-19], Nishida and Salomaa [16], Hie [13-14] and Honkala [6,8,10-12].) 

The following characterization of bounded context-free languages is due 
to Ginsburg and Spanier [5]. 
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Theorem 1. The family of hounded context-free languages is the smallest 
family of sets containing all finite sets and closed with respect to the following 
operations: 

(i) Finite union. 

(a) Finite product. 

(Hi) {x^y) * Z where x and y are words. 

(Here (x, y) * Z = x'Zy\) 

A new proof of Theorem 1 will be given in Section 4. Next, we define the 
class of power series corresponding to the class of languages considered in 
Theorem 1. 

Let Xoo be a fixed countably infinite alphabet and denote (see Kuich and 
Salomaa [15]) 

= {r e A <C >1 there exists a finite alphabet X C X^o 
such that r G A X* >}, 

= {r e ^{{x;^}} I supp(r) is finite}, 

= {tGA <C X^ >1 there exists a finite alphabet X C Xoo 
such that r G A^^ < X* »}. 

Now, let Bo be the least set S C A{{X^}} having the following properties: 

(i) C 5. 

(ii) If ri,r 2 G <S, then ri + r 2 G <S and rir 2 G S. 

(iii) If r E <S and wi,W 2 are words of X^, at least one of which is nonempty, 
then 

oo 

y^^w\rw2 G S. 

i=0 

Note that the class Bq depends upon the basic semiring A. 

The following theorem is equivalent to Theorem 1. 

Theorem 2. A series r G is ^-algebraic and s-hounded if and 

only ifre Bq. 

The following results due to Honkala [9] show that s-bounded algebraic 
series have nice decision properties. The proofs apply the theory of Parikh 
simplifying mappings and rely heavily on deep decision methods developed 
in Kuich and Salomaa [15]. 

Theorem 3. Suppose E is a computable field and X is a finite alphabet. It 
is decidable whether or not two given s-hounded series r,s G X* ^ 

are equal. 

Theorem 4. Suppose X is a finite alphabet and A = Z or A = Q. It is 
decidable whether or not a given s-hounded series r G <C X* is 

A-rational. 
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3 Characterization of s-bounded algebraic series 

Suppose A is a commutative semiring. To characterize s-bounded A-algebraic 
series we first define generic linear series. 

Suppose n is a positive integer and let Xn = {ci,dij,eij \ I < i,j < n} 
be an alphabet with n -h 2n^ letters. Then the proper A X* ^-algebraic 
system 

n 

J/i = Cj + ^ dijyjeji, l<i<n, (1) 

j=i 

is called a generic linear system of order n. The series r G A <C X* ^ is 
called a generic linear series of order n if r equals the first component of the 
strong solution of (1). 

Let (ri, . . . ,rn) be the strong solution of (1). The approximation sequence 
(r^)t>o associated to (1) is given by 

t 

— Cj + ^ ^ ^ dik-^dk^k2 • • • dk^_iku^ku^kuku — i ' ' ' ^k2k\^k\i’> 

l<ki^...^ku<n 

1 < z < n, t > 0. Hence the generic linear series r\ defined by (1) is given by 

oo 

n = Cl + ^ ^ dikidkiki ■ ■ ■dk^_iku^ku^kuku~i ■ ■ 

u=l l<ki,...,ku<n 

It is often useful to rewrite ri in the following way. For 1 < j < n, denote 

Ljt {d\ki djcik2 • • • ^ku — iku I 1 ^ J • • • ? l^u—1 ^ ^ ^ I}* 

Clearly, Lj is a regular language. Now, let a : X* — > X* be the isomorphism 
defined by a{dij) = eji, a{eji) = a(cj) = Q, 1 < z, j < n. Then 

n 

ri = Cl + wcja(wf (2) 

where a(w)'^ stands for the mirror image of a{w). 

To define closure under linear image we first define compatible morphisms. 
We say that the morphism h : A < X* > — > is compatible with 

the generic linear series of order n given by (2) if the following conditions 
hold: 

(i) There exists a word w\ G X^ such that h{dij) is -unary for all 1 < 
ij < n. 

(ii) There exists a word W 2 ^ X^ such that h(eij) is ii; 2 -unary for all 1 < 

hJ < n. 

(iii) For each i, I < i < n, the series h{ci) is quasiregular. For each pair (i, j), 
1 £ j £ f^he product 



h{dij)h{eji) 
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is qua^iregular. Furthermore, if h{dij) (resp. h{eji)) is not quasiregular then 
h{dij) = £ (resp. h{eji) = e). 

Note that if r is the generic linear series of order n and h is compatible 
with r, then h{r) € Now, a set 5 C A{{X^}} is said to be 

closed under linear image if h(r) € S whenever n > 1, r is the generic linear 
series of order n given by (2), h is compatible with r and, finally, h{ci) G S 
for all 1 <i <n. 

Next, let B be the least class S C such that S contains unary 

A-algebraic series and is closed under sum, product and linear image. Note 
that the class B depends upon the basic semiring A. 

We proceed to show that in many interesting cases B equals the semiring 
of s-bounded A-aJgebraic series. 

Theorem 5- Suppose A is a positive commutative semiring. Then r G 
A{{X^}} is A-algebraic and has a hounded support if and only if r G B. 

Proof. Suppose first that r G B. By Lemma 1, the set of A-algebraic se- 
ries having bounded supports is closed under sum and product. Clearly, this 
set contains also unary A-algebraic series. Suppose then that n is a positive 
integer, ri is the generic linear series of order n and /i is a morphism com- 
patible with ri such that h{ci), 1 < i < n, have bounded supports. Because 
h{ri) G it suffices to show that h{ri) has a bounded support. 

This follows by equation (2). 

Suppose then that r G has a bounded support. Then there 

exist a finite alphabet X, a positive integer m and nonempty words wi,. . . ^ 
Wm £ X* such that 

supp(r) Cwlw;...w*^. 

Let A = {ai,...,am} be a new alphabet with m letters and define the 
morphism g : A* — > X* by g{ai) = Wi, 1 < i < m. By the Cross-Section 
Theorem due to Eilenberg [3] there exists a rational language i? C a* 
such that g maps R bijectively onto W 1 W 2 .. .w’^. Define the series s G A 
A* > by 

s = g^^ir) © char(/?). 

Then s G A^^ <C -4”" and supp(s) C . . . aj^. Furthermore, g{s) = r. 
Because g is nonerasing we have g(B) C B. Therefore it suffices to show that 
s G B. We induct on m. 

If m = 1, the series s is unary and hence belongs to B. Suppose that 
m > 2. Without restriction we assume that s is quatsiregular. Let 

Vi Pij 1 ^ ^ ^ (3) 

be a proper A A* >*-algebraic system such that s equals the first compo- 
nent of the strong solution (si, . . . , Sn) of (3). Without loss of generality we 
assume that 

0 / supp(si) C ajoj . . . a;:;, 
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for all 1 < z < n. Without restriction we cissume also that in (3) for any 
1 < z < n, each variable in F = {yi, . . . ,z/n} has at most one occurrence in 
Pi. (If necessary, we add new variables and equations to (3).) 

Next, choose o, 1 < a < n, such that exactly a components of the strong 
solution of (3) have minimal alphabet A. We may assume that si,...,Sa 
have minimal alphabet A and Sa+i , . . . , have minimal alphabets smaller 
than A. By induction, Sa+i? • • • €: B. Now, let 

yi=Pi{yi,---,ya,Sa+l,---,Sn), 1 <*<£», (4) 

be the system obtained from (3) by deleting the last n — a equations and 
replacing each ^ > a, by sp in the first a equations. Then the righthand 
side of (4) is linear in Y' = {yi,...,2/a}- More precisely, there exist series 
U,Uij,Vij E A zi* >, 1 < z, j < a, such that (4) equals 

a 

Vi = U-^^ UijVjVji, I <i< a. 
j=i 

Here U € S, Uij € < aj » and Vij G < a;;;^ » for 1 < z, j < a. 

Furthermore, the series U and uijVji are quasiregular and, if Uij (resp. Vij) is 
not quasiregular, then Uij = e (resp. Vij = e), 1 < z, j < a. Let now 

a 

Vi — Q A ^ ^ 1 ^ ^ ^ 

J=1 

be the generic linear system of order a defining the generic linear series r of 
order a. Define the morphism h : A < X* >— > > by 

h{ci) = ti,h{dij) = Uij,h{eij) — Vij,l <i,j< a. 

Then h is compatible with r and 



Si = h{r). 



Hence s = si £ B. D 

A similar characterization now follows for all subrings of the real numbers. 
If A C R, denote A+ = {x € A | x > 0}. 

Theorem 6. Suppose A CH is a ring. Then r £ A{{X^}} is A- algebraic 
and has a hounded support if and only if r £ B. 

Proof. If r E B it follows as in the proof of Theorem 5 that r is A-algebraic 
and has a bounded support. Suppose then that r £ A^^S X* ^ has a 
bounded support where X is a finite alphabet. Suppose 



supp(r) C wlw 2 -..w:;^ 
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where m is a positive integer and wi,. . . ,Wm € X*. By Theorem IV 2.4 in 
Salomaa and Soittola [22] there exist series ri,r 2 € X* such that 

r = ri — r* 2 . 



Then 

r = r (D chdir{wlw 2 . . . w^) 

= ri © char(7i;*ii;2 . . . w^) — ri 0 oh^x(w{w2 • . . 

Because A+ is a positive semiring, Theorem 5 implies that the series ri 0 
didix{wlw 2 . • ^2 © chdj:{wlwT 2 . . belong to the class B when 

the basic semiring in the definition of B equals Aj^. Hence r G B where we 
now have A as the basic semiring. □ 

4 Language-theoretic applications 

Consider the classes Bq and B introduced in Sections 2 and 3, respectively. In 
general, Bq is a proper subclass of B. Indeed, all series of Bq are commutatively 
rational whereas B contains nonrational unary series. 

Next we give a new proof of the characterization of bounded context-free 
languages given in Theorems 1 and 2 due to Ginsburg and Spanier [5]. By 
Theorem 5 it suflBces to prove the next result. 

Theorem 7. If the basic semiring A equals the Boolean semiring, then Bq = 
B. 

Proof. We have to show that B C Bq. First, a unary series in is 

in fact in and hence in Bq. Because Bq is a semiring it remains 

to show that Bq is closed under linear image. Let n be a positive integer, 
r be the generic linear series of order n given by (2) and /i be a morphism 
compatible with r\. Suppose h{ci) G Bq for 1 < i < n. Factorize ft as a 
product ft = ft 2 fti by defining 



fti (^i) — ^i^h\{dij^ — h{dij^ , h\i^e%j) — hi^eij^, 1 ^ i,j ^ n, 

and 

h 2 {ci) = ft(ci) for I <i <n, h 2 {x) = x, otherwise. 

Then there exist sets /i © N^, 1 < i < n, and words W\,W 2 such that 

hiir)=Y^ w(ciw^. 

«=i 

Because fti(r) is B-algebraic, the sets /* are semilinear. Therefore 

Y w{h{a)w^ e Bo, 

U,k)eii 



( 5 ) 
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because h{ci) £ Bq, 1 <i <n. Indeed, if 

li = (mo, no) + N(mi,ni) + . . . + N(mt,nt) 
where mp,nfj € N for 0 < < t, then 

w{h{ci)w^ = 

U,k)€li 

OO OO OO 

jt=0 J2=0 ji=0 

Hence (5) holds true and, consequently, h{r) = h 2 hi{r) € Bq. □ 

As another interesting consequence of Theorem 5 we show that the char- 
acterization of Ginsburg and Spanier holds true even if multiplicities axe 
considered in the case of bounded context-free languages with finite degree 
of ambiguity. The following theorem states this in a precise way. 

Theorem 8 . Suppose the basic semiring A = N. If s £ has a 

bounded support and finite image then s £ Bo- 

In the proof of Theorem 8 we will use results concerning formal power 
series with commuting variables. Let X® be the free commutative monoid 
generated by X and c : A X* >> — > A <C X^ ^ be the canonical 
morphism. Denote 

^rat ^ ^c(r) | r e < X* »} 

and 

< X® »= {c(r) I r e < X* »}. 

The following lemma is due to Honkala [7]. 

Lemma 2. // r E <C X^ ^ has a finite image then r is a finite N- 
linear combination of series in X^ ^ of the form uvl -- - v* with 

pairwise disjoint supports. Here u,vi,...,Vp £ X^ and the Parikh vectors 
. . . , 'ip{vp) of the words , . . . , Vp, respectively, are linearly independent 
over Q. 

The proof of Lemma 2 relies heavily on deep results due to Kuich and 
Salomaa [15] and Semenov [23]. 

Proof of Theorem 8. Suppose s £ has a bounded support. 

By Theorem b, s £ B. We show inductively that all elements of B having 
finite images belong to Bq. First, if s is unary and has a finite image. Lemma 
2 implies that s E Sq. If s = si +S 2 or s = S 1 S 25 and s has a finite image, so do 
Si and S 2 . Hence we may assume inductively that si,S 2 £ Bq. Consequently, 
s E Bo- 
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In what follows we use the notation of the proof of Theorem 7. Suppose 
that s — h{r) and s has a finite image. Then h(ci), 1 < i < n, have finite 
images and we may assume inductively that h{ci) G Bq, I < i < n. Let 

n 

hi{r) = (6) 

i=l 



where supp(ti) C WiCiW^ for 1 < i < n. 

We want to show that h 2 {ti) G Bo for 1 < i < n. We consider first the 
series ti. li g : is a nonerasing morphism then g(Bo) C Bq- 

Therefore we may assume that w\ and W 2 are distinct letters. If h{ci) = 0, 
trivially h 2 {ti) G Bq. Otherwise, by (6), ti has a finite image. Furthermore, no 
two words in supp(ti) are commutatively equal. Hence also c{ti) has a finite 
image. Because c{ti) is N-algebraic, Lemma 2 implies that c{ti) is a finite N- 
linear combination of series of the form uv^ . . .Vp where u,vi, . . . ,Vp G X® 
and the Parikh vectors ^(vi), . . . ,^(vp) are linearly independent over Q. 
Furthermore, u contains the letter ci and vi,. . . ,Vp do not contain ci. Fix 
u,vi, . . . ,Vp. Let u = w{°CiW 2 ^ and vp = w\^W 2 ^ , 1 ^ £ P- Then h 2 {ti) is 

a finite N-linear combination of series of the form 

oo oo oo 

kp=0 k2=0 ki=0 

Hence, /i2(^i) G Bq. It is similarly seen that h2{ti) G Bo for 2 < i < n. 
Consequently 

n 

h{r) = h2hi (r) = ^ /i2(ti) € Bo- □ 

i=l 
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Full Abstract Families of Tree Series I 



Werner Kuich* 



Summary. We introduce full abstract families of tree series and show that their 
yields form full abstract families of power series that are closed under algebraic 
transductions. In the language case, the 3deld of a full abstract family of tree lan- 
guages forms a full abstract family of languages that is closed under context-free 
substitutions. 



1 Introduction and preliminaries 

Full abstract families of tree series (briefly, full AFTs) are families of tree 
series with certain closure properties. They are defined analogous to full ab- 
stract families of power series (briefly, full AFPs; see Kuich, Salomaa [19], 
Kuich [14]). These are a generalization of the full abstract families of lan- 
guages (briefly, full AFLs) as introduced by Ginsburg, Greibach [13] (see also 
Salomaa [20], Ginsburg [12] and Berstel [2]). Bozapalidis, Rahonis [7] intro- 
duced families of tree languages called sheaves. These sheaves are defined to 
have certain closure properties and are a generalization of the full AFLs in 
the framework of tree languages. The main difference of full AFTs in the 
language case and of sheaves is that the former are closed under linear non- 
deleting recognizable tree transductions (see Kuich [17]) while the latter are 
closed under alphabetic tree transductions (see Bozapalidis [4]). The relation 
of linear nondeleting recognizable tree transductions and of alphabetic tree 
transductions is yet unclear. 

The paper is divided in this and another two sections. The main result 
of Section 2 is that linear nondeleting recognizable tree transductions are 
closed under functional composition. In Section 3 we introduce recognizable 
tree cones and rec-closed families of tree series. Here a recognizable tree 
cone is a family of tree series closed under linear nondeleting recognizable 
tree transductions while a family of tree series closed under addition, top- 
catenation and least solutions of equations is called rec-closed. A family of 
tree series that is a recognizable tree cone and is rec-closed is called a full 
AFT. The main result of Section 3 considers the yields of the tree series of a 
full AFT: they form a full AFP. 

* Partially supported by the “Stiftung Aktion Osterreich-Ungarn” . 
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It is assumed that the reader is familiar with the basics of semiring theory 
(see Kuich, Salomaa [19] and Kuich [14], Section 2). Throughout the paper, 
(A, +,-,0, 1) denotes a commutative continuous semiring. This means: 

(0) the multiplication • is commutative; 

(1) A is partially ordered by the relation □: a C 6 iff there exists a c such 
that a + c = b, 

(ii) (A, +, •, 0, 1) is a complete semiring, 

(iii) di \ E C I, E finite), ai £ A, i G I, for an arbitrary 
index set /, where sup denotes the least upper bound with respect to □. 

In the sequel, we denote (A, +, •, 0, 1} briefly by A. 

Furthermore, E will always denote a ranked alphabet, where Ek, k > 0, 
contains the symbols of rank k and X will denote an alphabet of leaf symbols. 
By Tx:{X) we denote the set of trees formed hy E U X . If Eq ^ ^ then X 
may be the empty set. By A{{Ts{X))) we denote the set of formal tree series 
over Te{X) and by A{Tx;{X)) the set of polynomials. (For more definitions 
see Kuich [15].) 

Formal tree series induce continuous mappings called substitutions. Let 
Y = {2/ij ■ • • j2/n} denote a set of variables, and s = s{yi, . . . ,yn) and si, 
1 < ^ < n, be tree series in A{{Tx;{X U Y))). Then the formal tree series s 
induces a mapping s : {A{{Tjj{X U Y))))^ A{{Tx;{X U F))), whose value at 
si , . . . , Sn is defined as the substitution of the tree series Si for the variable yi, 
I <i <n, into the tree series s. It is denoted by s(si, . . . , Sn) or s[si/yi, 1 < 

1 <n]. (For a more detailed definiton see Kuich [15].) 

Our tree transducers will be defined by transition matrices. Let Yk = 
{2/ij • • • k > I, and F be sets of variables. A matrix M £ {A{{Tx;{X U 
Yk)))Y , k >1, r and I arbitrary index sets, induces a mapping 

M: (A((Ti:(XUF))))^><ix...x(A({T^(XUF))))^^i iA{{Tjj(XUY)))f 

(there are k argument vectors), defined by the entries of the resulting vector 
as follows: For Pi,. .. ,Pk £ {A{{Ts{X U Y)))Y^^ we define, for all i £ 

M{Pi,. . . , Pk)i = , • ■ ■ , {Pk)i,) = 

5 • • • J {Pk)ik) • 

Throughout the whole paper, I (resp. Q) will denote an arbitrary (resp. 
a finite) index set. 

2 Linear nondeleting recognizable 
tree transductions 

In this section we consider the linear nondeleting recognizable tree transduc- 
ers of Kuich [17] and show that the mappings defined by them are closed 
under functional composition. 




Pull Abstract Families of Tree Series I 



147 



In the sequel, U, S' and E” denote finite ranked alphabets, X, X' and X" 
denote leaf alphabets and Z = {zi | ^ > 1} denotes an alphabet of variables. 
We denote Zk = {zi\l <i < k}, k > I, and Zq = 0. 

A recognizable tree representation (with state set Q) is a mapping p, from 
E\J X into matrices with entries in A^^^{{Te>{X' U Z))) such that 

pi'.Sk^ {A^^^{{TE\X'UZk))))^^^\ k>\, 

: ToUX -> {A^^^{{Te'{X'))))Q>^\ 

Observe that, for / € A: > 1, /x(/) induces a mapping and 

{{A{{Ts'{X'yjZ))))Q^\((i{f) I / e 27)), {{A{{TE’{X'))))Q^\{^,{f) | / 6 27)) 

are X-algebras. Hence, the mapping p, : X {A{{Te>{X'))))^^^ can be 
imiquely extended to a morphism p : Te{X) (A((Ti:'(X'))))^^^ This mor- 
phic extension is defined by p{f{ti,. . . ,tk)) = p{f){p{ti ), . . .,p(tk)) for / E 
Xjfe, ti, ...,tk^ Te{X). For s E A{{Te{X))) we define p{s) = 

^p{t), where (8) denotes the Kronecker product (see Kuich, Salomaa [19], 
Section 4). 

A tree t E Te{X U Zk), A: > 1, is called linear iff the variable Zj appears 
at most once in t, 1 < j < A:. A tree t E Te{X U Zk), A: > 1, is called 
nondeleting iff the variable Zj appears at least once in t, 1 < j < A:. A 
tree series s E A{{Te(X U Zk))), A; > 1, is called linear or nondeleting iff 
all t E supp(s) are linear or nondeleting, respectively. A recognizable tree 
representation p is called linear or nondeleting iff all entries of p{f), f £ Sk, 
k > 1, are linear or nondeleting tree series, respectively. 

A linear nondeleting recognizable tree transducer (with input alphabet S, 
input leaf alphabet X, output alphabet X', output leaf alphabet X') 

T= (Q,p, S) 



is given by 

(i) a nonempty finite set Q of states, 

(ii) a linear nondeleting recognizable tree representation p with state set 

Q, 

(iii) S e {A{Ts'{Zi)Y^^ , where Sq = aqZi, Ug € A, q £ Q, called the 
initial state vector. 

The mapping ||T|| : A{{Te{X))) -4 A{{Te'{X'))) realized by a tree trans- 
ducer T = {Q,p, S) is defined by 

||T||(s) = 5(/i(s)) = 5]a, 

9€0 t£Ts(X) 

Before we show the main result of this section we need three technical 
lemmas. For these technical lemmas we have to extend the domain of a linear 
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nondeleting recognizable tree representation jj, from UUX to X\J XU Z] jj, is 
now a mapping from XUXU Z into matrices with entries in {X' U 

ZU Zq))), where Zq = {{zi)q | i > 1, q G Q}. The extension 

k>l, 



is defined by 

~ ^ ^ ^ Q G Q • 

Here Z^ = {{zi)q | 1 < « < A:, g E Q), > 1, and Z^ = 0. We call extended 

linear nondeleting recognizable tree representation. 

Let t E Ts{XUZk) (resp. t E Tx;>{X'UZ^)), k>0. Then we say that the 
variables Zi^^ , . . . ^ Zi^ E Zk (resp. ('2^ti)rij • • • ? ^ tti > 0, appear 

exactly once in t iff , . . . , zi^ (resp. {zi^ )ri , . . . , {zi^ )rm) appear once in 

t and the variables of Z - {zi ^ , • • • , (resp. Z^ - {(^iJn , • • • , {zim)rm}) 
do not appear in t. 

Lemma 2.1 Let ii be an extended linear nondeleting recognizable tree repre- 
sentation with state set Q, mapping XUXU Z into matrices with entries in 
{X'UZU Zq))) . Lett G Te{X UZk), k>0, be a linear nondeleting 
tree. Consider a tree t' E Te>{X' U Z^) that is in supp(/x(t)^), q £ Q. Then 
there exist states ri,...,rk GQ such that the variables (zi)n , • . . , {zk)rk ap- 
pear exactly once in t ' . 

Proof. Consider first a tree t E Ts{XUZk) such that the variables Zi^,..., Zi^ 
E Zk appear exactly once in t. Then we show by induction on the structure 
of these trees that there exist states ri , . . . , G Q such that the variables 
{Zi,)r , , . . . , {Zijrm appear exactly once in t'. 

(i) lft = x, the lemma is true. (Clearly, if no variable of Zk appears in t, 
the lemma is true.) 

(ii) If t = Zj for some zj E Zk we have p{t)q = (zj)q and the lemma is 
true. 

(iii) Let now t = /(ti, . . . ,tn), / ^ h,...,tn G Te{X U Zk), n > 

1. Then there exist mutually disjoint sets Ji, . . . , Jn with Ji U . . . U Jn = 

such that the variables zj, j G Ji, appear exactly once in U, 
1 < i < n. By induction hypothesis there exist, for each 1 < ^ < n, each 
qi G Q and each t[ E supp(/x(ti)^), states rj, j E Ji, such that the variables 
{zj)rj, j ^ Ji^ appear exactly once in t[. By Lemma 11 of Kuich [17], we 
obtain 

{tl > • • • ? l'n))q — P{f {zi , . . . , Zn)) q[p{l'i) / p{Zi), 1 ^ ^ ^ 

= 1 < * < n])[niti)/n{zi), l<i<n] 

~ S9i,...,9„eO M(/)9,(9i, -,9n)[M(^«)«i /^»> 1 < * • 

Hence, there exist states si,...,Sn G Q such that E supp(/x(/)^^(gj^...^ 5 ^) 
[p{ti)si/zi, I < i < n]). Since /^(/)g,(si is linear and nondeleting, we 
have completed the proof by induction and the lemma is proved. □ 
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We now define, for ri , . . . , € Q, the operator 

■ A{{Tr{X U Z^))} ^ A{{Ts{X U {(zi)., , . . . , (z*),, }))> 

as follows: For s e A{{Ts{XuZl^))) and t £ A{{Tr{Xu{izi)r„. . . , (zfc)r,})», 

{ (s, t) iff the variables {zi)n , . . . , (zk)rk 
appear exactly once in t, 

0 otherwise. 

Lemma 2.2 Let ^ he an extended linear nondeleting recognizable tree rep- 
resentation with state set Q and s £ A{{Tx:{X U Zk))), k > he a linear 
nondeleting tree series. Then, for q £ Q, 

Ks)q= ^ ¥>ri,...,r„iKs)q) ■ 

ri,.*7rfc€Q 

Proof. By Lemma 2.1, we have, for t £ Te{X U Zk) and q£ Q, 

ri,...,rfc€Q 

Hence, we obtain 

P(^)q — ~ 

^teTsiXuZk) Sri,...,rfc€Q(^’ - “ 

^ri,...,rfc€g “ Z^n,...,rfe6Q ' 

□ 

Compare the construction in the next lemma with the constructions in 
the proofs of Lemma 4.2 of Engelfriet [8] and of Theorem IV.3.15 of Gecseg, 
Steinby [10]. 

Lemma 2.3 Let pi be a linear nondeleting recognizable tree representation 
with state setQ\ mapping E\JX into matrices with entries in (X'U 

Z))). Furthermore, let p 2 be an extended linear nondeleting recognizable tree 
representation with state set Q 2 mapping E' \J X' U Z into matrices with 
entries in U ZU Zq^ff). Define the recognizable tree represen- 

tation p with state set Qi x Q 2 mapping EUX into matrices with entries in 
A^^^{{TEn{X" KJ Z))) by 

Pi^){quq2) = P‘2(Pl{^)qi)q2^ forx£EoUX, qi £ Qu ^2 ^ ^2 , 
p(f)(qi,q 2 ),iiri, 8 i),...,{rk, 8 k)) ~ 

^8i,...,8k (/^2(/^l (/)^i,(n,...,rfc))g2)[^l/(^l)«i J ■ • • 5 ^k ! j 

for f £ Ek, k ^ qi, r\ , . . . , r^ £ Qi, q 2 , s\, . . . , ^ Q 2 ■ 

Then p is a linear nondeleting recognizable tree representation and, for t £ 
Tz:{X) and qi £ Qi, q 2 ^ Q 2 , 
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Proof. By Corollary 13 of Kuich [17] and by the definition of 
. . . ,Sk G Q, /X is a linear nondeleting recognizable tree representation. 

The proof is now by induction on the structure of the trees t in T^{X). 

(i) For t e XqU X, the lemma is true by the definiton of /x. 

(ii) Let t = /(ti, . . . , / G Xn, h,.. . G Ts{X), n > 1. Then we 

obtain, for qi £ Qi, Q 2 ^ Qi, 

P2iPl{t)qi)q2 — 

/^ 2 (Z^y.i,...,r„€Qi • 5 f^l{^n)rn))q 2 ~ 

5])ri,...,rn€Qi P 2 {^^l{f)quiru^..,rn))q 2 [P 2 {^J>li^^^^^ I < i < n] = 

^ri,...,rn€Qi P‘^(P^(f)qi,(ri,...,rn))Q2[pi^i)(ri,8) / 5GQ] = 

Sri,...,r„€Qi €Q2 (/^2 (M1 (/)qfi ,(n,...,rn))92 ) 

[pi^i){ri,8] / 8j 1 ^ ^ ^ ^ ^ Q] ~ 

Sri,...,rn€Qi GQ2 (<p (/^2(/^l if)qi,{ri,...,rn))q 2 ) 

[zi/(zi)si, l<i< n])[fj,{ti)(^ri,8i)/zi, l<i<n] = 

^ri,...,rneQl I^Sl,...,«n6Q2 ,a2),((n ,Sl),...,(rn ,«n)) 

1 < X < n] == 

A^(/ (^1 ? • • • 5 ^n)){qi ,q2) ~ P(^){qi,q2) ’ 



Here the second equality follows by Lemma 11 of Kuich [ 17 ], the third equality 
by the induction hypothesis and the fourth equality by Lemma 2 . 2 . □ 

We are now ready for the main result of Section 2 . 

Theorem 2.4 Let /xi (resp. [12) be a linear nondeleting recognizable tree rep- 
resentation with state set Q\ (resp. Q2) mapping X\JX (resp. X' \JX') into 
matrices with entries in (((Ts' {X' U Z))} (resp. A^^^(lTx:"{X'' U Z)))/ 
Let Ti = {( 3 i,|Xi, 5 i) and T2 = (Q25 A^2, ‘S'2) be linear nondeleting recogniz- 
able tree transducers. Then there exists a linear nondeleting recognizable tree 
transducer^ such that |lT||(s) = l|T2||(||Ti ||(s)) for all s G A{{Tx;iX))). 

Proof. The linear nondeleting recognizable tree transducer T = {Qi x Q25/X, 
© 52) is defined by the linear nondeleting recognizable tree representation 
jj, of Lemma 2 . 3 . 

Let (^qiZ\^ ^qi ^ Ql ^ and (^^‘2)q2 bq^Zij bq2 G A^ 

qi e Qi- Then ( 5 i 0 S2)(q^^q^) = aq.bq^zi for qi G Qi, q2 G Q2- We now 
obtain, for s G A{{Tx;{X))), 

S«2€Q2 ^92 Y^t2eT^i (X')(^qieQi ®91 Sti 6 Te(X) (^> (^ 1)91 > ^ 2)^2 (^ 2)52 ~ 

^«i€Qi S»2€Q2 “9 i^« 2 IZt,eTi;(X)(®>*l)Z)t2€Tj;,(X')(^l(*l)9l>*2)/t2(^2)j2 = 
^(9i,92)€Qi XO 2 ®9i^ 92 Sti€Tr(X)(®>^l)/^(^l)(9i,92) ~ 

imiw- 



□ 
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3 Full abstract families of tree series 
and their yield 

In this section we introduce recognizable tree cones, rec-closed families of tree 
series and full AFTs, and consider their yields: The yield of a rec-closed family 
of tree series is a rationally closed family of power series; and the yield of a 
full AFT is a full AFP (closed under algebraic transductions). We consider 
this result on full AFTs to be a necessary condition that the definition of the 
notion of a full AFT makes sense. 

We make the following convention for the remainder of this paper: The set 
Eoo (resp. Xcyo) is a fixed infinite ranked alphabet (resp. infinite alphabet) 
and E (resp. X), possibly provided with indices, is a, finite subalphabet of 
Eoo (resp. Xoo); moreover, Eoo contains the symbols • and e of ranks 2 and 
0, respectively. Our basic semiring will be A((Tx'^ (Xqo))) and we define 

^{{^^oo(-^oo)}} = {5 G A{{Tj:^iXoo))) I there exist finite alphabets 
E C Eoo and X C Xoo such that s £ A{{Tx:{X)))} . 

Any nonempty subset of A{{Tx:^{Xoo)}} is called family of tree series. Fur- 
thermore, we define three subsemirings of A{{T^^(Xoo)}}, namely the fam- 
ilies of algebraic tree series (see Bozapalidis [6], Kuich [16]), of recognizable 
tree series (see Berstel, Reutenauer [3]) and of polynomials by 

A^g{|^T^^(Xoo)}} = {5 E A{{Tx;^{Xoo)}} I there exist finite alphabets 
X C Xoo and X C Xoc such that s E A^^^{{Tjj{X)))} , 
Arec{{T 27 ^(Xoo)}} = {« E (Xoo)}} | there exist finite alphabets 

X C Xoo and X C Xoo such that 5 E A^^^{{T^{X )))} , 
A{Xoo} = {s e A{{Tx;^{Xoo)}} I supp(5) is finite} . 

A mapping r : (-^ 00 )}} ^{{^i:oo(-^oo)}} is called linear non- 

deleting recognizable tree transduction iff, for 5 E A{{Tx'^(Xoo)}}nA((Ti;(X))), 
there exists a linear nondeleting recognizable tree transducer T with input 
alphabet X and input leaf alphabet X, such that r{s) = ||T||(5). 

For a family £ of tree series, we define 

9}l(£) = {t(5) I 5 E £ and r is a linear nondeleting 
recognizable tree transduction} . 

Observe that, by Theorem 2.4, 9Jl(9Jl(£)) = 97l(£). 

A family £ of tree series is said to be closed under linear nondeleting 
recognizable tree transductions iff £ = 9Jl(£). 

A family £ of tree series is called a recognizable tree cone iff it satisfies 
the following conditions: 

(i) £ contains a tree series 5 such that (s,x) = 1 for some x E Xo©; 

(ii) £ = 9Jl(£). 
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Condition (i) is needed to assure that the forthcoming Theorem 3.2 is valid. 
In the case of A = B, it can be replaced by the condition that £ ^ {0}. This 
definition and Corollary 14 of Kuich [17] yield the first result of this section. 

Theorem 3.1 is a recognizable tree cone. 

The next theorem shows that is the smallest recogniz- 

able tree cone. 

Theorem 3.2 Let £, be a recognizable tree cone. Then A^^^{{Tx;^{Xoo)}} C 

£. 

Proof. By definition, £ contains a tree series s such that (s, x) = 1 for some 
X € Xoo- Consider a recognizable tree series r and the linear nondeleting 
recognizable tree transducer T = {{q}, zi), where /x(x) = r, /x(x') = 0 for 
x' ^ X, x' G Xoo, and /j,{f) = 0, / G Uoo- Then l|Tl|(s) —r. □ 

For a family £ of tree series, we define the yield of £ to be 

yield(£) = {yd(5) | s G £} . 

Here yd is the yield mapping. Observe that yield(£) C i. e., yield(£) 

is a family of power series (see Kuich, Salomaa [19], Kuich [14]). Corollary 7.3 
of Kuich [15] (see also Bozapalidis [5]) yields the following result. 

Theorem 3.3 yield(A-'{{Ti;„(Xoc)}}) = 

Analogous to Bozapalidis, Rahonis [7] we introduce rec-closed families of 
tree series. A family £ of tree series is called rec-closed whenever the following 
conditions are satisfied: 

(i) 0 G £. 

(ii) If Si , 52 € £ then si 4- S 2 G £. 

(iii) If/G ^J(yQ IS of rank k ^ 0 and si , . . . , si^ G £ then f ^z \ , . . . , zj^ /'^i ? ■ • • 

. . . , Sk/zk] € £; if X G Xoo then x G £. 

(iv) If s G £ and x G Xoo then the least solution /xx.s of the equation x = s 
is in £. 

Hence, a family £ of tree series is rec-closed iff (£, -f, 0, (/ | / G Xoo U Xoo)} 
is a distributive multioperator monoid (see Kuich [15], Theorem 2.9) that 
satisfies condition (iv), i. e., our “rational” operations are 0, addition, top- 
catenation and least solutions of equations. (Observe that we do not ask 
for the closure under substitution as Bozapalidis, Rahonis [7] do for their 
REC-closed families of tree languages.) 

Theorem 3.4 (Xoo)}} is a rec-closed family of tree series. 

Proof. By Bozapalidis [5] and Kuich [15]. □ 

We are now ready to introduce full AFTs. We use the notation S^(£), £ 
a family of tree series, for the smallest rec-closed family of tree series that is 
closed under linear nondeleting recognizable tree transductions and contains 
£. A family £ of tree series is called full AFT iff £ = J?(£). 
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Theorem 3.5 (Xoo)}} is a full AFT. 

Theorem 3.6 (^oo)}} is a rec- closed family of tree series. 

Proof. By Corollary 28 of Bozapalidis [6] . □ 

We now show that the yield of a rec-closed family of tree series is a 
rationally closed family of power series. 

Theorem 3.7 Let SI be a rec-closed family of tree series. Then yield(£) is 
closed under addition^ multiplication and star and contains 0 and 1. 

Proof (i) Let ri , r2 G yield(£). Then there exist si , S2 E £ auch that yd(si) = 
Tt, i = 1,2. Sincer £ is closed under addition, s = si -h S 2 ^ £ and yd(s) = 
ri +r 2 ^ yield(£). Since £ is closed under top-catenation, s' = •( 51 , 52 ) E £ 
and yd(s') = rir2 E yield(£). 

(ii) Let s E £ and assume that x E Xoo does not appear in s. Consider 
the equation x = •(s,x) -he. Its least solution ^a:.(«(s,x) -he) is in £. Hence, 
the least solution /xx.yd(»(5,x) -h e) = jnx.(yd(s)x -h e) = yd(s)'‘ of x = 
yd(^(s,x) -h e) = yd(s)x 4- e: is in yield(£). Moreover, yd(0) = 0 and 0* = 1 
are in yield (£). □ 

We will show that, for a full AFT £, yield(£) is a full AFP. Here a full 
AFP is defined as follows. Let j(£), £ a family of power series, be the smallest 
rationally closed family of power series that is closed under rational trans- 
ductions and contains £. A family £ of power series is called full AFP iff 
£ = J(£). (See Kuich, Salomaa [19] and Kuich [14].) A lemma on algebraic 
representations and a theorem on algebraic cones are needed before the the 
main result of this section. (Kuich [14], below Theorem 7.10, gives the defi- 
nition of an algebraic representation.) 

Lemma 3.8 Let v he an algebraic representation defined by 
u : X {A^^{{X'*)))^^^ . 

Then there exists a linear nondeleting recognizable tree representation pL with 
state set Q x Q mapping XUX into matrices with entries in A^^^(^Ts’{X' U 
Z)f), 27' = {•,€}, such that, for all s E A{{Tx;{X))) and qi,q 2 E Q, 

yd(Ms)(,i,,2)) = i^(yd(s)V,,2 • 



Proof. We construct /x: 

(i) For X £ X and qi,Q2 €. Q vre construct according to Theo- 

rem 3.3 with the property that yd(/t(x)(,j_,j)) = 

(ii) For / e To and 91,92 G (? we define m(/)(„,,2) = Sg^,g^e, where 6 is 
the Kronecker symbol; hence, yd(/x(/)(,i,,2)) = Sg.^g^s = 

(iii) For / 6 Sk, k>l, and 9i >92, ?'i, •••)»■*, si, ... ,s* G < 5 , we define 

M(/)(9i,92)i((»'i,»i),. .,(ri,,«fc)) ^gi,ri^ai,r2 • • • ^ak-i,rk^Sh,q2 

• {Zl , •(« 2 , •(...• (zk-1 ,Zk).. .))) . 
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First, for a tree t £ Te{X) we show that yd(/i(t)(^i,^ 2 )) = ^{7^{^))qi,q2^ 
91 j ^2 ^ Q- The proof is by induction on the structure of trees in Te{X). 
The induction basis is true by (i) and (ii). Let now t = /(ti, . . . ,tk), f £ Xk, 
/? > 1, ti, . . . jtjfe G Tx;(X). Then we obtain, for qi,q 2 ^ Q, 

yd(/*W(,i.,2)) = yd(/i(/(<i,---,*fc))(9i,«2)) = 
y^QZn,...,rfc€Q I3 si,...,s*€Q M(/)(«i,92),{{ri,si),...,(rfc,Sfc)) 

[M(^l)(ri,«i)/’2l) • • • ) t^itk)(rk,Sk) / ^k]) — 
y*^(Sri,...,ri,€Q *^9i>fi<^*i,r2 • • • *^«fc-i,rt<^st,g2 

•(M(^l)(ri,«i) ) •(/*{^2)(r2,«2)> *(■••> l)(rfc_i,S)t_i) ! f^il'k)(rk,Sk)) ■■•)))) ~ 

yd{M(^i)(«.,«x))yd(M(*2)(s.,*2)) • • • 

. . .yd{li{tk-l)(sk-2,Sk-,))y^{f^(tk)isk-i,q2)) = 
E«x,...,«k-i€Q ''(yd(^i))«i.»i'^(yd(t2))«x,*2 ■ ■ ■ 

. . .Kyd{<fc-i))sfe_2,sfc-i*^(yd(«fc))sfc_i,,2 = 
*"{yd(ti) • • -yd(tfc)),i,,2 = '"(yd(t)),i,g2 • 

Hence, for s £ A{{Tx:{X))) and qi,q 2 G Q, 

yd(M(s)(,i,, 2 )) = Et€Ti:(X)(«>*)yd{/‘W(</i,92)) = 
Et€Ti;(X)(s>*)'"{ydW)9i,</2 = *"{yd(s))9i,92 • 



□ 

A nonempty family of power series is called algebraic cone iff it is closed 
imder algebraic transductions. Observe that each algebraic cone is a (rational) 
cone, i.e., a family of power series closed under rational transductions. 

Theorem 3.9 Let Z he a full AFT. Then yield(£) is an algebraic cone. 

Proof Let s £ 2, s £ A{{Tjj{X))), r = yd(5), and 3 = (Q, *S, P) be an alge- 

braic transducer. We will show that ||3||(0 G A{{X'*)) is again in yield(£). 
Observe that ||3||(r) = 5i/(r)F = E<,i,, 2 € 0 ^ 
q £ Q. By Theorem 3.3 there exist Sg,Pg £ A^^'^{{Ts'(X'))), 
•,e G Z", such that yd(s^) = Sq, yd{pq) = Pq^ q £ Q. By Lemma 3.8 there 
exists a linear nondeleting recognizable tree representation p with state set 
Q xQ such that yd{p{s) (^q^ ^q^)) = QiiQ 2 - Since £ is rec-closed, 

^91,92€Q*(®«1’*(/^W(91.92)>P92)) is in £. Hence, 

yd(E,i,92eQ*(«9l-*(Ms)(9i,92)>P92))) = 

I^9i, 92€Q yd(s9l)yd(/i(s)(9,,j2))yd(P92) = 

E}i,92€0 ~ l|3||(f) 

is in yield(£). □ 

Corollary 3.10 Let 2 be a full AFT. Then yield(£) is a full AFP that is 
closed under algebraic transductions. 
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Consider the macro power series introduced in Kuich [16], Section 6. They 
are the generalization of the 01 languages of Fischer [9] and the indexed 
languages of Aho [1] to power series. By Theorem 5.8 of Kuich [16], the 
family of macro power series is the yield of the family of algebraic tree series. 
In a forthcoming paper, Kuich [18], we will show that (Xoc)}} is 

closed under linear nondeleting recognizable tree transducers, i. e., is a full 
AFT. This result will imply that the family of macro power series is a full 
AFP that is closed under algebraic transductions. 

We now turn to the language case, i.e., our basic semiring is now given 
by ^(Tx-^(Xoo)) denotes the power set). We use without mentioning the 
isomorphism between ^{T£^{Xoo)) and (Xoo)))- 

A family £ ^ {0} of tree languages is called rec-closed iff (£,U,0, (/ | 
/ € i^oo C Xoo)) is a distribute multioperator monoid (see Kuich [15]) that 
satisfies the following condition: 

If L e £ and x E Xoo then the least solution fix.L of the tree language 
equation x = L is in £. 

Define J(£) to be the smallest rec-closed family of tree languages that is 
closed under linear nondeleting recognizable tree transductions and contains 
£. A family £ of tree languages is called full abstract family of tree languages 
iff £ = ^(£). 

We now connect our full abstract families of tree languages with full AFLs 
(see Salomaa [20], Ginsburg [12] and Berstel [2]). 

Theorem 3.11 Let £ be a full abstract family of tree languages. Then 
yield(£) is a full AFL that is closed under algebraic transductions. 

A substitution a is called context-free iff a{x) is a= context-free language 
for each x G X. 

Corollary 3.12 Let £ be a full abstract family of tree languages. Then 
yield(£) is a full AFL that is closed under context-free substitutions. 

Open question: What is the relation of full abstract families of tree lan- 
guages and sheaves (Bozapalidis, Rahonis [7])? 
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Linear Automata, Rational Series 
and a Theorem of Fine and Wilf 
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Summary. We consider partial functions from a free monoid A* to a field K 
and define a natural notion of periodicity in terms of linear semiautomata. We 
introduce the notion of greatest common divisor of two linear semiautomata and 
prove its existence and uniqueness. If a partial function is periodic with respect to 
two linear semiautomata Ai and A 2 then, under suitable conditions on its domain, 
it is also periodic with respect to the greatest common divisor of Ai and A 2 - As a 
corollary of this result one obtains a generalization of Fine and Wilf’s periodicity 
theorem and a stronger version of Eilenberg Equality Theorem for rational series. 



1 Introduction 

In this paper we shall present some recent results [12] that give a general- 
ization of the Fine and Wilf periodicity theorem in terms of linear automata 
and (partial) formal power series. 

A word is periodic of period p if it presents the same letter in any two 
positions which differ by p. The Fine and Wilf theorem states that if a word 
w has two periods p and q and \w\ > p + q — gcd(p, q), then w has period 
gcd(p,g) (cf. [11]. A weaker version of the Fine and Wilf [4] theorem can be 
stated as follows. Let m and t be two infinite words having periods p and q^ 
respectively. If m and t have a common prefix of length > p A- q — gcd(p, q), 
then m = t. 

The notion of periodicity can be defined for A-ary tree in terms of finite 
semiautomata. In fact, a finite semiautomaton, with an initial state qo, in- 
duces a periodicity on a tree requiring that any two nodes u, v {u and v are 
words over the branching alphabet of the tree), with qou — qov, must have 
the same label. 

Giammarresi et al. [5-7] introduced a natural notion of a greatest common 
divisor of two semiautomata, and proved that any tree which has two semi- 
automata as periodicities, under certain conditions on its domain, has also a 
periodicity in terms of the greatest common divisor of the two semiautomata. 

We will extend the previous results and notions to a different context in 
which one considers labels which are not letters but rational numbers or, more 
in general, elements of a field; in this way one can consider (partial) functions 
from a free monoid into a field K, i.e. formal series in non-commutative 
variables [1,14]. 
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As it will become clear in the paper, our notion of periodicity will lead 
us to consider rational series that have been widely investigated in Algebra 
and Formal Language Theory. Usually rational series are represented using 
multiplicity automata [3] or linear representations [1]. 

In this paper we consider linear automata and semiautomata [8,9,13] that 
give another representation of rational series. 

Considering the natural notion of morphism between linear semiautomata, 
we give a natural notion of divisibility between linear semiautomata. This 
notion allows us to define the greatest common divisor (gcd) of two linear 
semiautomata and prove its existence and uniqueness. We then give a notion 
of periodicity in terms of linear semiautomata using linear dependencies and 
we prove that if a function has periodicities in terms of two linear semiau- 
tomata then, under suitable conditions on its domain, it also has a periodicity 
in terms of the greatest common divisor of them. 

This result extends that of Fine and Wilf for words [4,11] and that of 
Giammarresi et al. for fe-ary labeled trees [5-7]. 

As a consequence of our result we can easily prove classical theorems 
of rational series theory, in particular the uniqueness (up to isomorphisms) 
of the minimal linear automaton of a rational series. Moreover, we give an 
improvement of the celebrated Eilenberg Equality Theorem [3]. 

In the last part of the paper we show how our main result can be seen as 
an effective generalization of Fine and Wilf periodicity theorem. In fact we 
will obtain a stronger version of this result that will be expressed in terms of 
linear recurrence relations. 



2 Rational series and linear automata 

We briefly present rational series [1,3,14]. Let K be a field and be the free 
monoid over the finite alphabet A. An application 5 : A* — AT is called a 
formal series with (non-commuting) variables in A and coefficients in K. We 
recall that a formal series 5 is called recognizable or rational if there exists 
a positive integer n, a row- vector A G a column- vector 7 G 

and a morphism n : A* w.r.t. the row-by-column product such 

that, for any w e A*, S{w) = Aju(u;) 7 . The triplet (A, ju, 7 ) is called a linear 
representation of S of dimension n. 

Now we introduce some notions on linear automata [8,9,13]. From now 
on, we will always consider vector spaces over K. 

A linear semiautomaton A of dimension n is a 4-tuple (Q, A, 5, go) where: 

• (5 is a vector space of dimension n; 

• A is a finite alphabet; 

• 6 : Q X A Q is a. transition function such that for any a G A, the map 

‘ Q Q, with Sa{q) = S{q, a), is a linear application; 

• go ^ Q is the initial vector. 
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Now, a linear automaton of dimension n is a 5-tuple {Q, A, S, qo,'ip) where 
(Q,A,6,qo) is a linear semiautomaton of dimension n and : Q -> AT is a 
linear application. 

We will sometimes denote a linear automaton as where ^ is a 

linear semiautomaton. We extend the application S on words in the usual 
way: S{q,e) = q, and for any w £ A* and a £ A 5{q,wa) = 6{8{q,w),a). 
Moreover, we shall always write qw instead of S{q,w) and qil^ instead of tl^{q). 
We can associate a formal series with any linear automaton. In fact, given A = 
(Q, A, S, qo, ^), we define the behavior of A as the formal series Sj\^ : A* K 
such that for any w £ A*, Sa{w) = {qow)i/j. Thus, we say that two linear 
automata on the same alphabet are equivalent if their behaviors coincide. 
Moreover, we say that a linear semiautomaton (Q, A, J, qo) is complete if the 
vector space Q is generated by the set {q^v \ v £ A*}, and then extend this 
concept to the corresponding linear automaton (A, 'ip). We remark that, given 
a linear automaton, it is always possible to construct an equivalent complete 
one. Consequently, from this point on, we will only concern ourselves with 
complete linear automata and semiautomata. Any linear representation of 
dimension n can be regarded as a linear automaton of the same dimension 
over the vector space . Conversely, one can easily prove that to any linear 
automaton A one can associate a linear representation (A, //, 7 ) of the same 
dimension such that the formal series S{w) = X/j,{w)^ coincides with the 
behavior of A . 

3 Greatest common divisor of linear semiautomata 

In this section we define the notion of greatest common divisor of two linear 
semiautomata and prove its existence and uniqueness. First of all we need to 
introduce a suitable divisibility relation in the class of linear semiautomata. 

Let A = (Q, A, J, go) be a linear semiautomaton and Q' a vector subspace 
of Q. We say that Q' is stable or invariant if 

for any a £ A and q £ Q', qa £ Q'. ( 1 ) 

We denote the quotient linear semiautomaton of A with respect to Q' as 
(Q/Q', A, (5', [go]) and define the transition function in the following way: for 
^ Q/Q' a £ A, (J'([g],a) = [g]a = [gaj. The stability condition 
assures that the function S' is well defined. Let us consider two linear semi- 
automata A = (Q, A, J, go) and B = (Q, A, J,po)- We can define a morphism 
from A to S as a homomorphism fi:om Q to Q such that qo^ = Po^ and for any 
q £ Q and a E A, {qa)(p = {q^)a. If such a homomorphism is both injective 
and surjective, then it is called an isomorphism and we say that the two linear 
semiautomata are isomorphic. We note that the inverse of an isomorphism be- 
tween linear semiautomata is an isomorphism too. We can extend the notion 
of morphism from linear semiautomata to linear automata in a natural way: 
if = (^i,V’i) = iQi,A,5i,qi,^pi) and B2 = (A2,ip2) = {Q2, A,S2,q2,tp2) 
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are two linear automata, we say that (^ : Qi Q2 is a morphism from Bi to 
B2 if is a morphism from Ai to A2 such that q'lpi := {q^)'ip2^ for q £ Qi. 
If such morphism is both injective and surjective, then it is called an isomor- 
phism and we say that the two linear automata are isomorphic. Also in this 
case, we remark that the inverse of an isomorphism between linear automata 
is an isomorphism too. 

We introduce now the notion of divisibility between semiautomata. Given 
two linear semiautomata A and B, we say that B divides A, and we write 
B/A, if there exists a surjective morphism from A to B. We remark that if 
Bj A and is an epimorphism from B to then B is isomorphic to the 
quotient of A through the invariant subspace Q' = ker((p). Let ^1, A2 be 
two linear semiautomata. We say that a linear semiautomaton B is a greatest 
common divisor of Ai and A2, and we write B = gcd(^i,.A.2), if B/Ai, B/A2 
and for any linear semiautomaton B' such that B' /Ai and B' /Ai^ one has 
S'/S. 

Proposition 1. The greatest common divisor of two linear semiautomata is 
unique up to isomorphisms. 

Let Ai = (Qi, A,(5i,gi), A2 = (Q2,A,S2,q2) be two linear semiautomata, 
with Qi and Q2 vector spaces of dimensions n and m, respectively. Let us 
consider the semiautomaton (Qi x Q2, A,(5i x ^2? (^i ?Q)) where the transition 
function is defined as follows: (p, q)a = {pa, qa), for any (p, q) £ QiX Q2 and 
a £ A. Let Q' be the subspace of Qi x Q2 generated by the set {{qi, -q2)w \ 
w £ A"^}. Obviously, Q' is stable. We can then consider the quotient linear 
semiautomaton 

B = {{QixQ2)/Q\A,6,[{quQ)])- 

We remark that for any [(p,g)] £ {Q\xQ2)IQ' and w £ A* , [{p,q)]w = 
[{pw,qw)]. Furthermore, we note that [(gi,Q)] = [(0,^2)]- In fact, (qi, 0 ) — 
(Qj 92) = (91, -92) € Q', that is (gi,Q) and (0,^2) are equivalent vectors. This 
fact trivially implies that 

[(9iw,0)l = [(0,g2w)], Vw € A*. (2) 

Proposition 2. Let Ai = {Qi,A, 5 i,qi), A2 = {Q2,A,S2,q2) be two linear 
semiautomata, with Qi , Q2 vector spaces of dimensions n and m, respectively. 
Then the linear semiautomaton B defined above is the greatest common divi- 
sor of Ai and A2 . 

Proof We will show that BjAi, omitting the proof for B/Ai which is sym- 
metric. In order to show that B/Ai, we will prove the existence of an epimor- 
phism between the two linear semiautomata. Let ip \Q\ (Qi x Q2)!Q' be 
the function such that qip = [(^,Q)] for any q £ Qi. The function ip is trivially 
a linear application between the two vector spaces. Moreover, q\ip = [(^1,0)], 
and for any q £ Qi and a £ A, {qip)a =[(g,0)]a =[(ga,0)] —{qa)ip. Therefore, 
^ is a morphism from A\ to B. The function ip is surjective, i.e. every element 
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in (Qi X Q2)/Q' has an inverse image through ip in Qi. In fact, since Ai and 
A2 are complete linear semiautomata, there exist two bases for Qi and Q2 
such that all the vectors of the base of Qi can be written 8 iS qiwi, ... ,qiWn, 
and those of the base of Q2 as q2Ui, . . . Then, by using Eq. ( 2 ), one 

can prove that for any [(p, g)] € (Qi x Q2)/Q\ there exists p e Qi such that 
[(PjQ)] = [(AQ)] = pip. It remains to show that B is the greatest common 
divisor of A\ and A2. Then, let = {Q,A,S,q) be a linear semiautomaton 
such that B' /Ai and B' IA2. That means that there exist two epimorphisms 
(p, ip with : Qi -> Oj 'ip ‘ Q2 Q such that qiip — q = q2'ip. We note that 
the last condition trivially implies that for any w e A*, {qi(f)w = {q2'ip)'w, 
that is 

for any w £ A*, {qiw)ip = {q2w)'ip. ( 3 ) 

If we show that B' /B^ the thesis follows. Consider x - ^ Q such that 

= pip A q'lp. The function x is a linear application since it is a linear 
combination of linear applications. We show that Q' C ker(x). Since any 
generator (qiw, —q2w) of Q' is such that {qiw, —q2w)x = (Qiw)ip — {q2w)'ip = 
0 (by Eq. ( 3 )), then we can conclude that all the linear combinations of 
the generators (that is the elements of Q') are in ker(x). We define now 
X • (Qi ^ Q2)/Q' ->• Q such that [(p, g)]x = (p^q)x- The function x is 
well defined. In fact, let (p, g) and {p',q') be vectors in the same class; then 
(p,Q)-{p',q') e Q' C ker(x). Thus, [(p,q)]x-[(p' ,q')]x = ip,q)x-(p' ,q')x = 
0 , that is [(p, g)]x = [(P^^ 0 ]X• We have shown that x does not depend 
on the particular element of the class. Moreover, one can check that x is 
a homomorphism of the two vector spaces {Qi x Q2)/Q' and Q. We can 
conclude that x is a morphism between the two linear semiautomata, in fact: 
[(gi,Q)]x = q, t/ivially; furtheremore ([{p,g)]o)x = [(pa,ga)]x = (pa)(p + 
{qa)xp = (pip)a + (q‘tp)a = ([(p, g)]x)o- We prove now that x is a surjective 
morphism. It suffices to show that any vector p of a base of Q has an inverse 
image by x- By the completeness assumption on Q, we can choose a base 
in which each element p is such that p = qu for some u £ A*. Therefore 
[(gi«,Q)]x = (gi«)<p + Q = {qi<p)u = qu=p. 

4 The periodicity theorem 

A natural notion of periodicity has been introduced for k-ary labelled trees 
[ 6 ]. A tree t is represented as a pair t = (T,/), where T is a prefix-closed 
set of words over A and / : T T is a map {F is the alphabet of the 
labels). The tree t = (T, /) has periodicity in terms of a finite semiautomaton 
with initial state qo if the following condition is satisfied: for any u,v £ T, 
qou = qov f{u) = f{v), i.e. two nodes that reach the same state must have 
the same label. 

Since we are considering linear semiautomata and partial functions from 
A* to K, it is quite naturcil to express periodicity in terms of linear depen- 
dencies. Thus, if ^ = (Q, A, 6 , qo) is a linear semiautomaton and S :T K 
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is a partial function (T C A*), we say that 5 is A-linear or A-periodic if for 
any wq, • • • , Un-i in T and ao, • . . , an-i in K, ctiQoUi = 0 implies that 

E"=o^ OiiSiui) = 0. 

Proposition 3 . Let A = {Q, A,S,qo) be a linear semiautomaton and S : 
T —^K be a map , with T C A* . The function S is A-periodic if and only if 
there exists a linear application ijj : Q ^ K such that S coincides in T with the 
behavior of the linear automaton {A,iIj), i.e. for any u ^T, S{u) = (go^)V^- 

We remark that a formal series S : A* K is rational if and only if there 
exists a linear semiautomaton A such that S is ^-periodic. We give now the 
notion of a complete set of words for two linear semiautomata by adapting 
an analogous notion for finite semiautomata [5-7]. 

We say that a subset P of A* is a complete set with respect to two linear 
semiautomata Ai = (Qi,A,Si,qi) and A2 = {Q2, A,S‘2,q2) if the subspace 
Q' of Qi X Q2 generated by the set {{qi,—q2)w | G A*} is generated by the 
set {(^1,— ^2)^ \ w £ P}. We remark that, given two linear semiautomata 
Ai and A2 , one can effectively construct a subset P of A* which is complete 
with respect to them. In fact it can be obtained by setting P = {e} and 
upgrading it as follows: for all w in P and for all a in A, if {qi,—q2)wa is 
not a linear combination of the vectors {(91,-92)^ \ u £ P}, then wa must 
be inserted in P. One can prove that the set P, so constructed, is complete. 
We remark that P is a prefix-closed finite set by construction. Moreover, 
fi:om the definition of Q' and the fact that the elements in Vp are linearly 
independent, we know that dim((3') = card(P). Since dim((Qi x Q2)/Q') = 
dim(Qi) 4- dim(Q2) ~ dim(Q'), one has 

card(P) = dim((3i) -f dim(Q2) - dim(((3i x Q2)IQ')- 

Lemma 1 . Let A\ = (Qi, A, and A2 = (Qi, A, ^2, g^2) be two linear 

semiautomata. If any A\ -periodic function is also Az-periodic, then linear 
dependencies are preserved, i.e. for any v,V{ E A*, E = 1 , . . . , n : 

qzv = aiq2Vi H + anq2Vn => Qiv = aiqiVi H -h anqiVn- 

Proposition 4 . Let Ai and A2 be two linear semiautomata. Then Ai/Ai if 
and only if any Ai -linear function is also A2-Unear. 

Theorem 1 . Let S : T K, with T C A* and let Ai and A2 be two 
linear semiautomata. If S is Ai-periodic and Ai-pcriodic and if T contains 
a complete set of words with respect to Ai and A2, then S is also B-periodic, 
where B = gcd(Ai, A2). 

Proof Let A\ = (Qi, A,(5i,gi) and A2 = (Q2, A,52,^2) with Qi and Q2 
vector spaces of dimension n and m, respectively. In order to show that B = 
((Qi X Q2)/Q^ A, S, [(g^i,Q)j) determines a new periodicity for the function 5, 
we have to define a linear application ^ : (Qi x Q2) /Q‘ K such that S{w) — 
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([(9ijQ)]^)V^ for any w e T. Since 5 is -periodic and ^2-periodic, there 
exist 'ipi : Qi ^2 : Q2 K such that for any w G T, S{w) = {qiw)il;i 

and S{w) = {q2w)'ip2- Then we define a morphism jp : Q1XQ2 K such that 
for any (p, q) G Q1XQ2, (p, = pipi -\~q'ip2- We show that Q' C ker(^), that 

is for any (p, q) G (p, q)'ip = 0. It suffices to show that this is true for the 
vectors of a base of Q'. Since T contains a complete set by hypothesis, we can 
choose a base such that each of its vectors has the shape {q\w^ — for some 
w GT. Then, {qiw^ —q2w)'ij) = {qlw)^pl — (q2'w)'ip2 = S{w) — S{w) = 0. This 
allows us to define ip : (Qi x Q2)!Q' -> K such that [{p,q)]'ip = (p, g)^- The 
function 'ip is well defined because it does not depend on the representative 
of the class. In fact, if (p, g),(p^gO ^re in the same class, we have that 
(P -P',Q- 9') G <3' (P - p',9 - 9') 6 ker(^) {p-p',q- q')i> = 0. 
Thus [(p, g)]V' - [(p', 9')]V» = (P, 9)^ - (p', 9')^ = {p-p',q- q')i> = 0. Since 
for any It; G T {[{qi,Q)]w)'ip — [{qiw,Q)]'ip = (qiw)'ipi = 5(u;), we can conclude 
that S is also 5-periodic. 

Corollary 1 . Let B\ = {Ai,ipi) and B2 = (- 42 ,^ 2 ) he two linear automata. 
Then they are equivalent if and only if their behaviors coincide on all the 
strings of length < dim(^i) + dim(^2) — dini(gcd(^i, ^2)) — T 

Proof If Bi and B2 are equivalent linear automata, their behaviors coin- 
cide for any w G A* and then the thesis trivially follows. Conversely, let 
A= {Q, A, S,q) = gcd(^i , A2) and suppose that the behaviors of the two au- 
tomata coincide on all the strings of length < dim(^i)-|-dim(^2)— dim(>l)-l. 
Let T be the set of all the words of length < dim(^i ) -f dim(w42) - dim(^) — 1 
and let S be the restriction of Sbi and then of Sb^ to T. The function S is, 
by hypothesis, .4i-linear and A2Aineax. Then, by Theorem 1, S is ^-linear, 
since T must contain a complete set with respect to A\ and ^2- 

Therefore, there exists a linear application ^p : Q K, such that for 
any u G T, {qu)'ip = S{u) = (gi^^)V^i = (g2^)^2- We observe that, since 
dim(>li) < dim(^i) -I- dim(^2) - dim(>l), the set T contains a complete 
set of words w.r.t. to A\. This means that for any word u G A* one has 
qiu = ^vqiy, for suitable At;’s in K. Since A divides Ai, by Lemma 1 
and Proposition 4 one also has qu = ^^g^- Prom the previous relations 

one derives that 

Sbi{u) = (9i“)V'i = ^ K{qiv)‘ii>i = ^ K{qv)tp = {qu)ip- 

v€T vGT 

Similarly one can prove that for any u G A* one has 5^2 (^) — 
then Sbi(u) = 5^2 (^)- 

The previous result can be seen as a stronger version of Eilenberg’s Equal- 
ity Theorem. In fact, this theorem states that two linear automata Ai and 
A2 are equivalent if and only if their behaviors coincide on all the strings of 
length less than or equal to dim(^i) -h dim (^2) — 1- The following result is 
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well known. It is the analogous for linear automata of the theorem on the 
uniqueness of the minimal linear representation of a rational series (cf. [ 1 ]). 

Corollary 2. Let S be a rational series. The minimal linear automaton of 
S is unique up to isomorphisms. 

Proof. Let Bi = and B2 = (^2,V^2) be two minimal linear automata 

of S. Let Ai = (Qi,A,Si,qi) and A2 = (Q2,A,62,q2) and A = {Q,A,6,q) = 
gcd(>li,^2)* By the Theorem 1 the formal series S is Alinear, i.e. there 
exists a linear application 'll; : Q K such that S is t he behavior of the linear 
automaton {A, 'll;). Prom the fact that dim(>t) < dim(^i), dim(^) < dim(^2) 
and the hypothesis of minimality of B\ and B2, we can derive that dim(.A) = 
dim(w 4 i) = dim(^2)- Moreover, we know that there exist an epimorphism 
If from Ai to A and an epimorphism (p' from A2 to A; since A, Ai, A2 
have the same dimension, p and p' are isomorphisms, and then Ai and 
A2 are isomorphic. It remains to show that, if (j) denotes such isomorphism, 
9V1 = ( 9 ' 0 )V^ 2 , for any q' £ Qi. 

It suffices to prove this condition on a base for Qi. To this aim, one can 
consider a set P of words such that {git/; j ly E P} is a base for the vector 
space. Then, as 5 is the behavior of the linear automaton {A,'ij;)j for any 
u G A* {qu)'ij; = (giu)t/;i = (g2^^)V^2; this implies that {qiw)'ipi = {q2'w)'^2 
for any w G P, and then (git/;)V'i = {{qiw)(j))'il;2 for any w G P, where 
Q2W = {qiw)(f> by definition of morphism between semiautomata. Thus, our 
claim holds. 

5 Linear semiautomata on polynomials 

In this section we deal with linear semiautomata in which the transition func- 
tions are extended so to allow processing polynomials in non-commuting vari- 
ables over an alphabet A, that is the formal series S : A* K whose support 
Supp(iS) ={w G A* I S{w) ^ 0 } is a finite set. Let us denote the set of all such 
polynomials as K{A) and consider a linear semiautomaton A = {Q,A^S, go). 
We can extend by linearity the transition function over K (A) in the following 
way: for any q gQ and XiUi G K (A), g(XliLi '^*^0 ~ Xiqui. Also 
in this case we are going to consider complete semiautomata. We remark 
that, if A is a complete semiautomaton, then the vector space Q is equal to 
{goP I P € K{A)}. We note that for any p,r G K{A), one has qoip-r) = {qop)r. 
We can consider the following set of polynomials associated with the semi- 
automaton A: 

Ia = {p\ qop = fl}- 

The following result holds. 

Proposition 5. Given a linear semiautomaton A, the set is a right-ideal 
for K {A). 
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One can prove that the right-ideal associated with a linear semiautomaton 
characterizes its class of isomorphism. 

Proposition 6. Let A and B be two linear semiautomata. Then, A and B 
are isomorphic if and only if = Ib- 

We try to express the notion of greatest common divisor in terms of right- 
ideals. At this purpose, we recall that the sum of two right-ideals h and h 
is the set /i 4- /2 = {pi + Pi \ P\ ^ h,P 2 ^ h}- Trivially, the sum h -f h is 
the smallest right-ideal containing both Ii and h. 

Proposition 7. Let A = {Qa^ o.'nd B = (Qb, A, Qq, 6') be two linear 

semiautomata and let C =gcd(^, B) . Then, Iq = . 

Proof. By Proposition 2, the semiautomaton C is of the form {{Qa x Qb)/Q'^ 
A, [(^OjQ)] = [(Qj^o)])- F^^st we show that Ic Q Ia-^ By definition, for 
any p £ Ic we have that [(go,0)]p = [(goP, 0)] = [(0,0)], that is (goP, 0) £ Q'. 
Since Q' = {(go,— go)^ I ^ ^ AT(A)}, there exists a polynomial p such that 
(goP, Q) = (goP, — goP)- By this equation one has that gop = goP, that is p—p = 
p£ Ia and that 0 = -q^p, that isp£ Is - This implies that p = p+p £ 
Conversely, we show that Ia + ^b ^ Bor any pi £ I a and P 2 € Z^, it holds 
that [(go,Q)](pi+p2) = [(9 o^i+P 2),Q)] = [(9oPi,0)]+[(goP2,0)] =[(goP 2 ,Q)] = 
(by Eq. (2) generalized oveTK(A)) = [(Q, g(,p 2 )] = [(II, fl)], that is pi +P 2 £lc- 

6 Linear recurrence relations 

In this section, as application of the previous results, we give a proof of 
a theorem on finite sequences satisfying two linear recurrence relations (cf. 
Theorem 3). This result, which is a generalization of Fine and Wilf’s theorem, 
seems to be known to people working on the theory of Fade approximants. 

We can transpose on linear recurrence relations the results obtained in the 
previous sections. This is achieved by showing that we can always identify 
a linear recurrence relation with a linear semiautomaton over a one-letter 
alphabet {x} and conversely. 

Given a semiautomaton over a one-letter alphabet A = (0, {x}, J, go), 
we will represent A by the triplet (<5,go,^), where (p : Q -> Q is the linear 
application defined as follows: for any q G Q,q(p = qx. A linear recurrence 
relation of degree n is defined as a sequence of n coefficients (oq, • • • ^ocn-i) 
in K that we will denote by a, setting also deg(a) = n. Let I be either the 
set of natural numbers N or an interval [0, h] of N. We say that a sequence 
{U)i€i of elements of K satisfies the linear recurrence relation a if 

n— 1 

tn^i = ^2 for any I > 0 and n + i G Z. 

j=o 
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We show that a linear semiautomaton A = {Q, of dimension n can 
be associated with a linear recurrence relation (qq, . • . ,an-i) of degree n 
and conversely. In fact, given the linear semiautomaton A of dimension n 
over a one-letter alphabet {x}, since A is complete, one can prove that the 
set {qo,Qo^, • • • is a base for the vector space Q; hence, since the 

vector qox'^ is a linear combination of the elements of the base, there exist 
coefficients a^'s in K such that q^x"^ = ' Consequently, the 

linear recurrence relation (qq, . . . ,an-i) can be associated with the linear 
semiautomaton A. 

On the other hand, given a linear recurrence relation (qq, . . . ,o:n-i) oi 
degree n we can define the associated linear semiautomaton as {K'^,qQ,(p) 
where go = ( 1 ? 0 ? • ■ • ? 0 ) ^ and for any g E q(f = qM with 
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Proposition 8 . Let J- 




and A‘2 


be two linear semiautomata 



alphabet. Then A\ and A2 are isomorphic if and only if they can be associated 
with the same linear recurrence relation. 



One can easily prove the following result. 

Proposition 9 . Let a = (oq, • • . ,0:^-1) be the linear recurrence relation as- 
sociated with a linear semiautomaton A. Then for any linear map ijj : Qa 
K, the behavior of the linear automaton (A^'ip) is a rational sequence {an)n>o 
of elements in K that satisfies the linear recurrence relation (ao> • • • 5<^n-i)* 

In the sequel we will denote by Aa the linear semiautomaton associated 
with a linear recurrence relation a. We remark that if we consider the function 
S :T ^ K where T ={e, x, . . . , x^”^}, that is a prefix-closed subset of {x}*, 
the sequence {S{x^))i=o,...,k-i satisfies the linear recurrence relation a if and 
only if 5 is >l^-periodic. 



From the results of the previous section linear semiautomata on one- 
letter alphabet {x} process polynomials in the commutative ring K[x]. We 
remark that in this context the distinction between right-ideals and left-ideals 
falls, so that we will always deal with (two-sided) ideals. From polynomials 
rings theory, we know that every ideal I of K[x] is principal, that is there 
exists a polynomial p E / such that I p >= {pr \ r E K[x]}, and that 
such a polynomial is of minimal degree in the ideal. We will show that this 
polynomial has a special form in the context of linear recurrence relations. We 




Linear Automata, Rational Series and a Theorem of Fine and Wilf 



167 



first recall the notion of characteristic polynomial of a given linear recurrence 
relation A = (Ao, . . . , Ap_i), that is 



px{x) = x^ - \p-ix^ ^ - . . . - Ao. 

Obviously deg(A) = deg(pA)- Moreover, we remark that, if qo is the initial 
state of then, by definition of linear recurrence relation associated with a 
linear semiautomaton, it holds that q^px — q^x^ —qo\p-.ix^~^ — . . .—qo^o = Q, 
i.e. the characteristic polynomial of a linear recurrence relation always belongs 
to the ideal associated with the relative semiautomaton. 

Proposition 10. Let X be a linear recurrence relation and A\ its associ- 
ated linear semiautomaton. The ideal I^x is generated by the characteristic 
polynomial p\ of X. 

Proof. As well known, every ideal I of K[x] is principal and is generated by 
a polynomial of I of minimal degree. By construction A is such that p\ is a 
polynomial of minimal degree satisfying qop\ = 0. Then p\ is a polynomial 
of minimal degree in the ideal Iax ? ^d thus the thesis follows. 

Theorem 2. If Ay = gcd(^A, -4;,), then p^ = gcd{p\,Pp). 

Proof. By Proposition 10 we know that p\, Pp generate the ideals 
I Ax j respectively. Therefore, by Proposition 7, we can state that <p^ >— 
= Jax + =< Px> <P^l >=< PAjPm >■ From a well known result 

about Euclidean rings, we can state that p^ = gcd(pAjPM)- 

Thus we obtain a generalization of Fine and Will’s periodicity theorem. 

Theorem 3. Let X and p be two linear recurrence relations with associated 
characteristic polynomials px and p^ respectively ^ and let p^f =gcd(pA>P/z)- If 
a sequence of elements in K satisfies X and p and is longer than or equal 
to deg(pA) H- deg(p^) — deg(p^), then it satisfies also the linear recurrence 
relation 7 relative to p^ . 

Proof Let A = (Aq, . . . , An-i), p = {po,. . . , pm-i) and 7 = (70, . . . ,7^-1). 
One can consider a sequence {to? • • ■ satisfying the two linear recur- 
rence relations A and p, with k — d. Let T = {e, x, . . . , x ^~^ } and let 

S \ T Ff be a partial function such that S{x^) = U, for i = 0, 1. The 
set of words T contains a complete set with respect to Ax and Ap and then, 
by Theorem 1, S is ^^-linear with Ay = gcd(^A? By Propositions 3 and 
9, one has that S satisfies the linear recurrence relation 7. Moreover, by The- 
orem 2, it holds that 7 is associated with the polynomial py that is equal to 
gcd{px,Pn)- Thus, one can conclude that S is -linear, with 7 = gcd(A,p), 
that is the sequence {to, • ■ ■ , satisfies 7. 
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Remark 1. We observe that the previous result is well known if one considers 
an infinite sequence of elements of K satisfying two linear recurrence equa- 
tions. This follows ft-om the fact that the characteristic polynomials of the 
linear recurrence relations satisfied by a given infinite sequence form a right 
ideal of K[x] (cf. [2]). However Theorem 3 gives an extension of this result 
to finite sequences in the spirit of Fine and Wilf’s periodicity theorem. 

Now, suppose that a sequence is p-periodic and g-periodic; such a sequence 
satisfies the two linear recurrence relations A =(1, 0, . . . , 0) with deg(A) = p 
and p = (1, 0, . . . , 0) with deg(p) = q. We have that p\ = — I and = 

x^ — 1. Since gcd(x^ ~ — 1) = x^ — 1, where d = gcd(p, g), Fine and 

Wilf’s periodicity theorem is a particular case of Theorem 3. 
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Summary. Evolutionary grammars form a grammatical model for the evolution of 
genomes where the basic derivation steps correspond to mutation in genomes. The 
generated language consists of all words which can be generated from the axioms 
(a finite set of words) by iterated derivation steps. 

In this paper we study the number of axioms, the number of mutations and 
the size of the mutations which are necessary in order to generate some languages. 
Especially we show that these parameters cannot be bounded in order to generate 
all possible languages. 

Furthermore we study the function which gives the number of words which can 
be generated by a certain number of derivation steps. We present some bounds and 
examples for such functions. 



1 Introduction and Definitions 

Grammars can be considered as language generating devices. The classical 
operation iteratively performed in order to obtain the words of the language 
from the axiom is the (sequential or parallel) substitution of some subword (s) 
by (an)other word(s). 

In the last years there have been introduced some other types of gram- 
mars (or systems) motivated by considerations from molecular genetics. The 
starting point is the fact that DNA and RNA structures, genomes and chro- 
mosomes can be described to a certain extend as words, e.g. a DNA strand 
can be presented as a word over the alphabet of the compound symbols 
(A,T), (T, A), (C, G), (G, C). Further, the basic operations in a derivation 
step model mutations occuring in the evolution of genomes and/or chromo- 
somes. Thus the grammars (or systems) obtained can be considered as a 
model for parts of the evolution. 

In the evolution the genome of an organism mutates at the level of indi- 
vidual genes by point mutations and at the genome level by some large-scale 
rearrangements in one evolutionary event. The most well-known rearrange- 
ments are deletion, inversion, translocation and duplication. 

• Deletion cancels a segment of a chromosome. 

• Inversion replaces a segment of a chromosome with its reverse DNA se- 
quence. 
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• Translocation moves a segment to a new location in the genome. 

• Duplication copies a segment of a chromosome. 

A formalization of these mutations in terms of formal language theory 
was first given in [ 11 ]. Results on the power of these operations and relations 
between them can be found in [ 13 ], [ 2 ] and [ 6 ]. 

On the basis of the above large-scale reaarrangements a grammatical 
model for the evolution of genomes was presented in in [ 4 ] and further studied 
in [ 5 ]; a context-sensitive variant was considered in [ 3 ]. Instead of rules for 
replacements (as in Chomsky grammars or Lindenmayer systems) finite 
sets C, /, T and D of words are given; these sets contain the words which 
can be deleted (cancelled), reversed, translocated or duplicated, respectively. 
Any derivation step corresponds to an application of a deletion of a word 
fi:om C or of a reversal of a word from / or of a translocation of a word from 
T or of a duplication of a word from D. The language consists of all words 
which can be generated by a finite sequence of derivation steps starting with 
a word from the a given set A of axioms. In biological terms, the language 
consists of all genomes which can be obtained from a given set A of genomes 
by a finite sequence of certain given mutations. 

We now give the formal definition of a context-free evolutionary grammar 
and start with some notations. Throughout the paper we assume that the 
reader is familiar with basic notions of the theory of formal languages. We 
refer to [ 9 ] and [ 10 ]. 

For an alphabet V, hy V* (V"^) we denote the set of all (non-empty) 
words over V. The empty word is denoted by A. For a word w e V* and 
a letter a € V, #a(^) denotes the number of occurrences of a in ly. The 
reversal it; ^ of a word w GV* is inductively defined by 

= A and (vx)^ = xv^ for x GV\v gV* . 

The cardinality of a (finite) set M is denoted by #(M). For two functions / 
and g from the set N of natural numbers into N, we write /(n) = S{g{n)) if 
/ and g are asmptotically equal. 

A context-free evolutionary grammar is a system 

G = {V,A,CJ,T,D), 

where V is an alphabet and A, C, 7 , T and D are finite languages over V. 

For x,y GV* we write: 

X1ZX2, y = X1X2 for some xi,X2 G C, 

X1ZX2, y = xiz^X2 for some xi,x>2 ^ V*,z G 7 , 

X1ZX2XZ, y = X1X2ZXS for some xi,X2,xs G V* or 
X1X2ZXS, y = X1ZX2XS for some xi,X2,xs G V*,z G T, 
X1ZX2, y = X1ZZX2 for some xi,X2 € V*,z G D. 



X => y in X = 
X y iS. X = 
X => 2/ iff a: = 

T ^ 

X — 

X => 2/ iff X = 




Numerical Parameters of Evolutionary Grammars 



173 



We write x y if and only if x y for some X E {(7, /, T, D}] the 
reflexive and transitive closure of is denoted by 
The language L{G) generated by G is defined by 

L{G) = {w \x =^w for some x ^ A) . 

The language contains all words which can be generated by iterated derivation 
steps. 

In this paper we study the number of axioms, the number of operations 
and the maximal length of the words inCU/UTUD which are necessary in 
order to generate some languages. Especially we show that these parameters 
cannot be bounded in order to generate all possible languages. 

Furthermore we study the differentiation function which gives the number 
of words which can be generated in a certain number of derivation steps. We 
present some bounds and examples for such functions. 

2 Descriptional Complexity 

In this section we consider the descriptional (syntactic) complexity of lan- 
guages generated by context-free evolutionary grammars. We are interested in 
the minimal number of axioms and operations, respectively, and the maximal 
length of the words associated with an operation. Formally, for a context-free 
evolutionary grammar G = (V, A, C, /, T, D), we set 

a{G) = #{A), 

o(G)=#(C) + #(/) + #(T) + #(D), 

1{G) = max{|iy| \ wGC\JIUTUD} 

and extend these measures to a language L generated by a context-free evo- 
lutionary grammar by 

a{L) = min{a(G) | L = L(G),Gis a context-free evolutionary grammar}, 
o(L) = min{o(G) | L = L(G)^ Gis a context-free evolutionary grammar}, 
1{L) = min{Z(G) | L = L(G),Gis a context-free evolutionary grammar}. 

Theorem 1. A language L is finite if and only if o{L) = 0. 

Proof Let L be a finite language, and let V be the set of symbols occurring 
in at least one word of L. Then L = L(G) for the context-free evolutionary 
grammar G = (V, L, 0, 0, 0, 0). Since o{G) = 0 we obtain o(L) = 0. 

If o{L) = 0 for some language L, then there is a context-free evolutionary 
grammar G = (V, A, 0,0, 0,0) with L = L(G). By L(G) = A, G generates a 
finite language which proves that L is finite. 
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The measure o(G) corresponds to the number of productions in a (usual) 
Chomsky grammar. The context-free languages form an infinite hierarchy 
with respect to the number of productions (see [7]). Rirthermore, the measure 
1{G) corresponds to the radius of a H system which is grammatical device 
based on splicing as another mutation. With respect to the radius the lan- 
guages generated by H systems form an infinite hierarchy, too (see [8]). In this 
section we shall prove analogous assertions for the measures for context-fi:ee 
evolutionary grammars introduced above. 

Theorem 2. For any measure d G {a, o,Z} and any natural number r > I, 
there is a language L generated by a context-free evolutionary grammar such 
that d{L) = r. 

Proof. We consider the language 

n 

L Li where Li = {{ba^b)'^ | m > 0} for 1 < i < n . 

i=l 

Because L is generated by the context-free evolutionary grammar 

G = ({a, 6}, {ba^b 1 1 < i < n}, 0, 0, 0, {6a*6 | 1 < i < n}) 

with a{G) = n, o{G) = n and 1(G) = n -h 2, we obtain 

a(L) < n, o(L) < n and l(L) < n -h 2. (1) 

Now let us assume that H = (V, A, C, /, T, D) is a context-free evolution- 
ary grammar with L(H) = L. If there is a derivation w' w in H with 
w' e Li, w e Lj, 1 < i,j < n and i ^ j, then there also is a derivation 
w'ba^b wba^b. Since w'ba^b £ Li C L and wba^b ^ L, we get a contradic- 
tion. Thus, for any i, 1 <i <n, AnLi has to be a non-empty set. Therefore 
a(H) > n for any context-free evolutionary grammar H with L(H) = L 
which implies a(L) > n. By (1), we obtain a(L) = n. 

Let a = max{|z| | z G A}. We consider a word w e Li, I < i < n, with 
\w\ > a -f 1. Let w = (ba^by. By the length of w there is a word w' £ L 
with w' w and w' ^ w. By the above considerations w' £ Li, too, say 
w' = (ba^b)^ for some k. 

If w' w or w' w, then \w'\ = \w\ and hence w' =w in contrast to 

j T I I I I 

the choice of w' . 

Let us assume that w' w. Then w' = W\XW 2 and w = wixxw 2 for 
some wi,W 2 ^ V* , X £ Thus 

H^a{x) = - #a('w') = i(l - k) and #b(x) = 2(1 - k ) . 

Therefore 

X £ \r,s >0,r s = i}U {{ba^bY^^} . 
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If X = a^bba^, r,s > 0, r + s = i, i.e. / = A: -h 1, then we can apply x G D to 
ba^bba^b which yields ba^^~^b and hence n = i. If, in addition, r > 0 and s > 
0, we can apply x G D to ba^~^bba'^~~^bba^~^b and obtain ba^^~‘^~^bba^~^b 
from which i = n—1 follows. This contradicts i = n. Thus x = a^bba^ implies 
r + 5 = n and r = 0 or s = 0. 

We now define 



Mj = {a'^b{ba^bYba^ | r, s > 0, r 4- s = j, t > 0} U {{ba^bY 1 1 > 0} 
for 1 < j < n, 

Mn = {a'"b{bd^bYba^ | r, s > 0, r + s = n, t > 0} U {a^bb, bba'^} 
U{{ba^bY \ t>0}. 

By the considerations above, we get x G Mi. 

Let w. Then w' = wixw2 and w = wiW 2 - By analogous arguments 

we can show that x G Mi, again. 

Thus (C U iP) D Mi ^ 0 for 1 < z < n. Furthermore, Mi H Mj = 0 for 
1 ^ j ^ i ^ j. Therefore C L\ D contains at least n elements and 
o{H) > n holds for any context-free evolutionary grammar H with L{H) = L. 
Hence o(L) > n. By (1), o{L) — n. 

Moreover, for 1 < z < n, |x| > 2 -h z holds for any x G (CUD)r\Mi. Thus 
1{H) > n -h 2 for any context-free evolutionary grammar H with L{H) = L. 
Therefore 1{L) > n -h 2 and, by (1), 1{L) = n -f 2. 

Hence the statement holds for d G {a, o}, r > 1 and d = /, r > 3. It is 
ecisy to see that = 1 and l{{a^}^) = 2, and therefore the statement 

holds in the remaining cases, too. 

In order to get a finer measure than the number of all operations given 
by o{G) = #(C') 4- #(/) 4- #(T) -f #(D) one can consider the quadruple 
0{G) = (:^(C),#(/), #(T),#(D)). The investigation of this measure is left 
as an open problem. 



3 The differentiation function 

The notion of a differentiation function of a grammar was firstly introduced 
in [1] for deterministic tabled Lindenmayer systems. It presents a measure 
for the number of objects which can be derived in a given grammar by a 
given number of derivation steps. Formally we obtain the following notion 
for context-free evolutionary grammars. 

Let G = {V,A,C,I,T,D) be an evolutionary grammar. Then we define 
the its differentiation function 

/g:N-4N by foik) = card{Lk{G)). 
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Example 1. We consider the context-free evolutionary grammars 

Gi = ({a,6},{aa},0,0,0,{aa}), 

Ga = ({a,6},{aa},0, {aa},0, {aa}) , 

Gs = ({a,6},{aa6},0,{aa},{6},{aa}). 

Then, for k >1, 

Lfc(Gi) = {a2*+2}, 

LkiGz) = 

LkiGs) = {a’’6a* | r + s = 2i, 1 < i < A:} U , 

and thus 

foi (A:) = 1 for k>l, 
foi (k) = k + l for k>l, 

fo^ik) =3 + 5 + ...{2k + l) + l = {k + lf for k>l. 



We only show the statement concerning Lk{G^), the modifications for the 
other cases are obvious. 

From the axiom aab of Gs we can generate by inversion of aa the word 
aab, again, by translocation of b the words baa, aba, aab and by duplication of 
aa the word aaaab. Thus the statement holds for k = 1. 

Now let w G LkiGs). By induction hypothesis, w — a^ba^ with r s = 2i 
for some i,\<i<k or w = a^^^^^b. 

We first consider the former case. Since the translocation and inversion 
does not change the number of occurrences of a and b, we obtain by these 
operation a word a^ ba^ with r' -f 5 ' = 2i. If we applied the duplication of 
aa we get a^~^^ba^ or a^ba^~^^. Because r -h 5 -i- 2 = 2(i 4- 1), in all cases the 
generated words have the desired form. 

In the former case we generate from w a word of the alba^ with r s = 
2Jfc 4- 2 = 2{k -h 1) or and all words have the desired 

form, again. 

Moreover, these considerations also show that all words of the desired form 
are contained in Lk+i{Gz)- 

We now give an upper bound for differentiation functions of context-free 
evolutionary grammars. 

Theorem 3. For any context-free evolutionary grammar G, there are con- 
stants Cl and C ‘2 such that fc (k) < ci • cf for k > 1. 

Proof. Let G = {V, A, C, I, T, D) be an evolutionary grammar. We set 

d = max{|u| | u G D} , a = #(A) , b = max{|i;| 1 G A} . 
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Then, for any k >0 and any word in ^ G Lk{G), \z\ < b k • d. Thus 

foik + 1) < #{w \ \w\ <b + kd} 

b-\-kd ^ 

= E = ^(y) _ 1 • ((#(V))'+*‘'+' - 1) 

By setting ci = and C 2 = (#(V^))'^) the assertion follows. 

The following shows that the exponentially upper bound is obtained for 
some context-free evolutionary grammars. 

Theorem 4. For any natural number c, there is a context-free evolutionary 
grammar G such that foik) = for k>0. 

Proof. Let c be given. We consider the context-free evolutionary grammar 
G = (V, {a%a^}, 0, 0, 0, {aW \l<i<c}. 

By induction on k we prove that 

Lk{G) = | • - U ^ . . . c}} 



which implies 

fa{k) = #(L*(G)) = c* 

for A; > 0. 

By definition, Lq{G) = {a^ba^} and therefore the statement holds for 
* = 0 . 

Now let w G Lk^i{G). Then => w for some w' G Lk{G). By induction 
hypothesis w' = Let w be derived from w' by 

duplikation of some a^6a*, 1 < i < c, where the duplication involves the j-th 
occurrence of b. Then 

w' = 

= 

= 

which proves that w has the desired form. 

If we apply in succession the duplications of 
such that in any step the la;St b is involved, then we get a^ba^^^^ba^^^^b 
. . G L^+i(G). Hence L^ 4 .i(G) contains all words of the consid- 

ered form. 

Thus the induction statement is shown for A: -h 1. 
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We now prove two assertion concerning the closure of the family of dif- 
ferentiation functions under operations. 

Lemma 1. Let f and g he two differentiation functions of context-free evo- 
lutionary grammars. Then their sum f g is a differentiation function, too. 

Proof. Let f = fo and g = fn for two context-free evolutionary grammars 
G = (Vg,Ag,Cg,Ig,Tg,Dg) and H = {Vh,Ah,Ch,Ih,Th,Dh)- Without 
loss of generality we can assume that Vg H V/f == 0. Then no operation from 
H can be applied to an element of L(G), and conversely, no operation from 
G can be applied to an element of L(H). Moreover, L(G) U L{H) = 0. We 
consider the context-free evolutionary grammar 

G' = (Yg U Vh, Ag U AhjCg U Ch^ Ig U Ih,Tg U Th,Dg U Dh) • 

By induction we can show that Lk{G') = Lk{G) U Lk{H) which implies 

fcr'ik) = fG{k) 4- fnik) = (/ + g){k) 



by the disjointness of the languages generated by G and H. 

Let / and g be two function from N into N, then their convolution f * g 
is defined by 

k 

if * 9){k) = ^ fit) ■ 9{k - i) . 
i=0 

We now show an analogon of Lemma 1 for convolutions instead of sums. 

Lemma 2. Let f and g he two differentiation functions of context-free evolu- 
tionary grammars. Then their convolution f *g is a differentiation function, 
too. 



Proof. Let f = fG and g = fn for two context-free evolutionary grammars 
G = (Vg^Ag^Cg^Ig^Tg^Dg) and H = {Vh,Ah,Ch,IhiTh,Dh)- Without 
loss of generality we can assume that VgOVh = 0 - We consider the context- 
free evolutionary grammar 

G' = {Vg U Vh, Ag • Ah,Cg U Ch, Ig U Ih,Tg U Th,Dg U Dh) • 

By induction we show that 

Lfc(G') = Lo(G) . Lk{H) U Li(G) • Lk-i{H) Li • • • U L,(G) • Lo{H) . (2) 

By the construction of G', the statement holds for k = 0 . Let w G Lk{G'). 
Then there is an integer i, I < i < k, such that w = wiW2 for two words 
wi G Li{G) and W2 G Lk-i(H). Let us apply an element x G Gg U Ch to 
w.li X £ Cg, we have to apply x to wi and get w -- W\W2 => w[w2 = w' 
where w[ G Lk+i{G). Hence 



w' e Li+xiG)Lk-i{H) = Li+,{G)L^k+i)-{i+i){H) C Lk+i{G') . 
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Analogously, x G Ch implies 



w' € Li{G)Lk-i+i{H) = Li{G)L^k+D-i{H) C Lk+i{G') . 



Furthermore, it is easy to see that any word from Li(G)Lk^i{H) can be 
generated in k steps, more precisely, by i steps according to G and k — i steps 
according to H. 

By the supposition on disjointness and (2), 

k k 

fo'ik) = #{Lk{G')) = Y. = Y fG{i)fH{k - i) 

i=0 i=0 

k 

= Y1 f(^)9(k - i) = if * 9){k) . 
i=0 

We use the Lemma 2 to show that asymtotically any polynomial can be 
obtained as a differentiation function. 

Theorem 5. For any natural number there is a cgeg G such that /g(^) = 
9{k^). 

Proof. For n G {0, 1, 2} the statement follows from Example 1. Let us assume 
that there is already a context-free evolutionary grammar G with /g(^) = 
0(k^). Then we construct the context-free evolutionary grammar G' with 
fo' = fc * foi j where Gi is the context-free evolutionary grammar from 
Example 1 (see the proof of Lemma 2. Then 

fG-{k) = Yfo(i) = &(k^-^^)- 

i=0 

Thus the statement follows by induction. 

The presented results give some upper and lower bounds for and some 
examples of differentiation functions; the characterization of the family of 
differentiation functions of context-free evolutionary grammars is left as an 
open problem. 

Finally we present two clcisses of context-free evolutionary grammars with 
differentiation functions bounded by a constant or linear function. 

Lemma 3. If G = then there is a constant c such that 

foik) < c for k >0. 

Proof Obviously, because the set of duplications is empty, L{G) is finite. Let 
c be the cardinality of L{G). Then fcik) = #{Lk{G)) < #{L{G)) = c. 
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Example 1 shows that there are context-free evolutionary grammars with 
a non-empty set of duplications which also have a differentiation function 
bounded by constant. 

Before we present the other class we give the definition and a property of 
a number-theoretic function. For a set A of natural numbers we define the 
function 

k 

fA(k) = #({^ ai,. I at. e A for 1 < j < k}) . 

J = 1 

Lemma 4. Let A = {ai, U 2 , . . . , a^} with 0 < ai < a 2 < as <...< ag and 
m = max{af+i — ai\l<i<s — 1}. Then 

^ • fcj + 1 < fA{k) < {ag - ai) • k + l. 
m 

Moreover, both bounds are optimal. 

Proof. Obviously a\ • k and Og-k axe the minimal and maximal number which 
can be obtained by addition of k numbers of A. Hence any sum S of interest 
satisfies aik < S < Ogk. This implies the upper bound. 

We now prove that any interval li = [aiA: -f irn,aik H- (i 4- l)m), 0 < 

^ < I (as-ai)k I _ 1 contains at least one sum of k numbers of A. Obviously, 
this holds for i = 0 by aik € Iq- Now let c = be the maximal 

number in Ii which can be represented by sum of k numbers of A. Since 
* < — 1 we obtain c < aik + “ 1)™ < Thus there 

is a r such that < Og. Let a,v = aj- Then we consider the sum 

r— 1 k 

c' = +a;+i + ( ^ Uij) 

j=l j--r+l 

of k numbers of A. Because c' = c -h (a^+i - a/) < c -h m and c is maximal in 
Ii, we obtain c' G /j+i. Since we have intervals and the additional 

sum Ogk (which belongs to no interval), the lower bound follows. 

The optimality of both bounds follows by considering A = {1, 2, 3, . . . , s} 
or A = {m, 2m, 3m, . . . , sm} (for some m). 

Lemma 5. For any context-free evolutionary grammar G = ({a},{a^},0, 
I,T,D) where D contains a non-empty word (i.e. the underlying alphabet of 
the grammar is unary, there is only one axiom, no deletion and at least one 
non-empty duplication), fci^) = 0{k). 

Proof First we assume that /UT = 0. The application of v £ D to a'^ leads 
to Thus taking A = {|i;| | v G D}, we obtain fA{k) = /g(^) for A: > 1. 

Hence the statement follows from Lemma 4. 

Now let lUT ^ 0 and X ^ D. We set r = min{|z| | z G lUT}. If we apply 
an inversion or a translocation to a word w, then vj is not changed by this 




Numerical Parameters of Evolutionary Grammars 181 



operation because the underlying alphabet is unary. To any word w € Lk{G) 
with \w\ > r we can apply an inversion or a translocation, and thus w is 
also contained in Li{G) for I > k. Moreover, if D does not contain the empty 
word, all words from Lr{G) have a length > r. It is easy to see that 

fAu{ 0 }(k) < fa{k + r) < #(Lr(G))/>iu{o} W for A: > 1 . 

Now the stement follows from Lemma 4. 

The proof for the case / U T ^ 0 and \ £ D follows analogous arguments 
and is left to the reader. 
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Summary. With motivations from various areas, in several places mechanisms 
based on iterated (non-deterministic) finite state sequential transducers were con- 
sidered. It is known that such mechanisms can characterize the family of recursively 
enumerable languages. We continue here the study of such devices, investigating the 
hierarchy on the number of states. We find that this hierarchy collapses: four states 
are enough in order to characterize the recursively enumerable languages, three 
states cover the ETOL languages, while two states can cover the EOL (hence also 
context-free) languages. The case of deterministic transducers remains open. 



1 Introduction 

Iterated gsm (generalized sequential machine) mappings were already investi- 
gated in seventies (see, e.g., [11], [5], [9]), especially in relation with L systems 
area. Because, as it is easy to see, such a device can simulate the work of a 
type-0 Chomsky grammar, in this way we obtain characterizations of recur- 
sively enumerable languages. (Depending on the precise mode of defining the 
accepted language, we need or not an intersection with T*, where T is the 
alphabet of the desired language, as a final squeezing operation.) 

The phenomenon of repeated translations of a text appears in many areas, 
especially in linguistics and in literary studies. We refer to [3] for bibliograph- 
ical details. 

A recent area where the need of iterated gsm’s has appeared is DNA 
computing, in relation with the so-called computing by carving, [6]. We will 
briefly present this idea in the next section. In short, we “compute” a language 
by identifying strings in its complement and “carving” them from a given 
superlanguage; the removed strings are obtained by iterating a gsm on a 
starting regular language. 

In this framework, it appears as a natural problem to consider simple 
gsm mappings, for instance, with a bounded number of states. The problem 
was formulated in [6] whether or not the number of states induces an infinite 
hierarchy of the languages computed by carving. We solve this problem here. 
Somewhat surprisingly, this hierarchy collapses at a rather low level: three 
states suffice. We place this question in a more general framework, that of 
languages obtained by an iterated use of a sequential transducer. We find 
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that (non-deterministic) iterated transducers with four states characterize 
the recursively enumerable languages (the extra state is needed in order to 
select strings which are terminal with respect to a given alphabet; in the case 
of carving this is accomplished in a different way - see details in the sections 
below). Several connected results are also given (three states suffice in order 
to cover the ETOL languages, while two states suffice in order to obtain all 
EOL languages). 

These results deal with non-deterministic gsm’s. The case of iterated de- 
terministic gsm’s remains almost completely open. 



2 Computing by Carving 

DNA computing is a new field of research, trying to make use of the huge 
paralellism and other nice features of the structure and behavior of DNA 
molecules. The first practical step in this respect was the experiment reported 
in [1], where a small instance of the Hamiltonian Path Problem was solved 
by biochemical means. Another successful experiment was reported in [4]: 
finding the size of the maximal clique in a graph. Information about further 
developments in this area can be found in [7] and [8] . 

The procedures in [1] and [4] follow the same phases (on similar strategies 
are based other experiments of DNA computing): (1) one first generates a 
laxge set of candidate solutions to the problem, (2) one identifies sets of 
non-solutions, and (3) one removes such non-solutions from the candidate 
set, repeatedly, until only the solutions remain. In [4] one explicitly speaks 
about the complete data pool from which, by specific filtering procedures, 
one removes the non-solutions. 

Inspired by these observations and by the (theoretically) easy way of im- 
plementing the difference of languages in DNA terms, the general strategy of 
computing by carving was proposed in [6], proceeding along the three steps 
mentioned above. In short, in order to construct the set of the solutions to a 
problem, we first construct a superset of it (one which is “easy” to be con- 
structed) and then we filter it iteratively (by “carving” sets of non-solutions, 
also “easy” to be constructed) until only the solutions of the problem remain. 

In some sense, this idea is not new at all: this is exactly the style of the 
Erathostene’s sieve for constructing the prime numbers, where the “complete 
data pool” is the set of natural numbers (which is given “for free” in advance). 

For the formal language theory, the computing by carving leads to a rather 
new way of identifying a language L: start from a superlanguage of L, say 
M, large and easy to be obtained, then remove from M strings or sets of 
strings, iteratively, until obtaining L. Contrast this strategy with the usual 
“positivistic” grammatical approach, where one produces the strings in L at 
the end of successful derivations, discarding the “wrong derivations” (deriva- 
tions not ending with a terminal string), and constantly ignoring/avoiding 
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the complement of L (for elements of formal language theory which we will 
use here we refer to [10]). 

In particular, M above can heV*, the total language over the alphabet V 
of the language L. Therefore, as a particular case, we can try to produce V* - 
L, the complement of L, and to extract it from V*. As one knows, the family 
of recursively enumerable languages is not closed under complementation. 
Therefore, by carving we can ^‘compute^^ non-recursively enumerable langua- 
ges ! 

However, identifying a language L by generating first its complement, H = 
V* - L, and then computing the difference F* - i/ is a rough procedure. A 
more subtle one, present also in the DNA computing experiments mentioned 
above, is that where L is obtained at the end of several difference operations, 
possibly at the end of an infinite sequence of such operations: we construct V * , 
as well as certain languages Li , L 2 , . . ., and we iteratively compute V* - Li, 
{V* — Li) — L‘ 2 ,. . . , such that L is obtained at the limit. 

Of course, the sequence of languages Li,i > 1, must be defined in a 
regular, finite, way. The proposal in [6] is to consider regular sequences of 
languages: 

A sequence Li, L 2 , . . . of languages over an alphabet V is called regular if 
Li is a regular language and there is a gsm g such that Lj+i = g{Li),i > 1. 

Then, a language L CV* is said to be C-REG computable if there is a 
regular sequence of languages, Li,L 2 , . . . (that is, a pair (Li,g) as specified 
above), and a regular language M CV* such that L — M - (Ui>i ^i)- 

We can denote by g^ the ith iteration of g,i > 0, and then Lj+i = 
9 ^{Li),i > 0 (taking g^{H) = i/ for all if C V*). Denoting by g*{Li) 
the union ^ “ M ~ g*(Li). We are again back 

to the complement with respect to M of a single language, g*{Li), but this 
language is the union of the languages in a sequence defined by finite tools: a 
regular grammar - or a finite automaton - for the language L\ , and the gsm 

9 )’ 

The following result is proved in [6] : 

Theorem 1. A language is C-REG computable if and only if it can be writ- 
ten as the complement of a recursively enumerable language. 

Consequently, the family of C-REG computable languages contains all 
context-sensitive languages and it is incomparable with the family of recur- 
sively enumerable languages (CS is closed, but RE is not closed under com- 
plementation). 

The following necessary condition is proved in [6]. 

Lemma 1. If (Li,g) identifies an infinite language L = g*{Li), then there 
exists a constant k such that for each x £ L we can find y E L such that 
|x| < |y| < fc|x|. 
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For instance, the language L = {a^ I ^ > 1} does not have this property. 
Therefore, RE includes strictly the family of languages of the form for 

L regular. However, we have the next result (proofs can be found in [11], [5], 

[9]): 

Theorem 2. Every recursively enumerable language L CT* can be written 
in the form L = 5^*({ao}) fl T*, where g is a gsm - depending on L - and ao 
is a fixed symbol not in T, 

At the end of [ 6 ] one asks the question whether or not the number of states 
in a gsm induces an infinite hierarchy of C-REG computable languages. We 
shall solve this problem in the next sections, by proving that the answer is 
negative. 

3 Iterated Sequential Transducers 

We first introduce a general set-up for our investigations, by defining the 
notion of an iterated finite state sequential transducer (in short, an IFT), as 
a language generating device. Among the reasons for considering this notion 
are the facts that, according to Theorem 2.2, in order to characterize RE 
by iterated gsm’s we can start fi:om a single symbol, but we also need an 
intersection with a language T*. Such an intersection can easily be done by 
a gsm (for instance, using a special state for this purpose), so we introduce 
this feature “inside” the gsm work. Because we explicitly provide the starting 
symbol ao in the machinery, we prefer to use a new terminology, speaking of 
IFT’s and not of gsm’s of a special form and with a special functioning. 

An IFT is a construct 7 = (AT, F, 5o,oo,F, F), where K^V are disjoint 
alphabets (the set of states and the alphabet of j), sq G K (the initial state), 
ao eV (the starting symbol), F C K (the set of final states), and P is a finite 
set of transition rules of the form sa -> xs', for s,s' G K,a £ V,x £ V* (in 
state s, the device reads the symbol a, passes to state s', and produces the 
string x). 

For s,s' £ K and u,v,x £ V*,a E F we define: 

usav h uxs'v iff sa — > xs' £ P. 

This is a direct transition step with respect to 7 . We denote by \~* the reflexive 
and transitive closure of the relation h. 

Then, for w,w' £ V* we write 

w ==> w' iff sqw h* w's, for some s £ K. 

We say that w derives w'\ note that this means that w' is obtained by trans- 
lating the string w, starting from the initial state of 7 and ending in any 
state of 7 , not necessarily a final one. We denote by the reflexive and 
transitive closure of the relation ==^. 
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If in the writing above we have s ^ F (we stop in a final state), then we 

write instead of =^; that is, w w‘ iff s^w h* ly's, for some s £ F. 
The language generated by 7 is 

-^(7) = ^ V* \ ao =>* w' w, for some w* £ V*}. 

Therefore, we iteratively translate the strings obtained by starting fi'om 
ao, without care about the states we reach at the end of each translation, 
but at the last step we necessarily stop in a final state. This makes possible 
the avoiding of the intersection with a language of the form of T*, as in the 
statement of Theorem 2.2. 

The IFT’s as defined above are non- deterministic. If for each pair (s, a) £ 
K xV there is at most one transition sa xs' in P, then we say that 7 is 
deterministic. 

We denote by IFTn^n > 1, the family of languages of the form L{j), for 
non-deterministic 7 with at most n states; when using deterministic IFT’s we 
write DIFTn instead of IFTn. The union of all families IFTn, DIFTn, n>l, 
is denoted by I FT, DIFT, respectively. 

We will compare these families with the families REG, CF, CS, RE (of 
regular, context-free, context-sensitive, recursively enumerable languages, re- 
spectively), and with OL, EOL, ETOL (of languages generated by interaction- 
less, by extended interactionless, and by extended tabled interactionless L 
systems, respectively); DOL, DEOL are the families corresponding to OL, EOL 
generated by deterministic L systems. 

Convention. When comparing the power of two generative devices, Gi , 
G2, the empty string is ignored: Gi is equivalent to G2 if and only if L{Gi) — 
{A} = L(G2) - {A}. 

4 The Hierarchy IFTn Collapses 

The following relations are direct consequences of the definitions: 

Lemma 2. (i) /FT„ C IFT„+uDIFT„ C DIFT„+u n > 1. (ii) DIFTn 
C IFTn, n>l, DIFT C IFT. (iii) IFT C RE. (iv) IFTi = OL, DIFTi = 
DOL. 

Lemma 3. EOL C IFT 2 . 

Proof. Consider an EOL system G = {V,T,w,P) (the total alphabet, the 
terminal alphabet, the axiom, and the set of rewriting rules). Let oo,ai be 
two new symbols. We construct the IFT 

7 = ({5o,5i}, VU {ao,ai},so,ao,{si},P'), 

P' = {so^o CLiwso, soai aiso} U {soa xsq \ a x £ P} 

U {so^i -> si} U {sia asi \ a £ T}. 
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The first step, soao h aiwso, introduces the axiom of G and the left marker 
a\. Each subsequent translation soaiz h* giz'sq precisely corresponds to an 
equivalent derivation step z => z' in G. The final state si can be reached only 
by a transition soaiz h siZ] the work of 7 can be finished only when z GT*. 
After removing the symbol ai, the final state si cannot be reached again, 
hence no further iteration can modify the string. Consequently, L{G) = L(^). 

□ 

As a consequence of this proof we get the inclusion CF C IFT2. This 
result is also proved in [9] (Theorem 1.7) for iterated gsm’s (using an inter- 
section with T* as a squeezing mechanism). A characterization of context-free 
languages is also obtained in [9]: two-state iterated gsm’s such that a copying 
cycle exists for each state generate only context-free languages (a copying 
cycle is a cycle using a rule of the form sa -> as, that is leaving unchanged 
the symbol a). In view of the previous lemma, the copying cycle property is 
crucial in this characterization. 

The IFT 7 constructed above is not deterministic, even when starting 
from a deterministic EOL system G: we have the rules sqUi aiso and 
soUi -> Si with the same left hand member. We do not know whether or not 
DEOL C DIFT2, but still we have: 

Lemma 4 . DIFT2 -CF^^, DIFT2 - OL ^ 0 . 

Proof. Consider the DIET 7 = ({so,5i}, {oo,a,6},so,ao, {5i},P), with 
P ~ {sqUo — y a6asi, sqcl — y osq, sob — y bsi, sia — y aasi}. 

We obtain 

L(j) = {aba^^ I ^ > 0}- 

This is obviously a non-context-free language. It is also non-OL: in order to 
generate strings with arbitrarily large suffixes we need a rule of the form 
a — > a*, z > 2; using this rule for rewriting the leftmost occurrence of a in any 
string from L(j) we get a string not in L(j). □ 

The hierarchy IFTn.n > 1, is not an infinite one. This fact is not unex- 
pected: we know from [11], [5], [9] that iterated gsm’s characterize the family 
RE^ which can be written as RE C IFT\ start the proof of this inclusion 
from a universal type-0 grammar^ in this way we obtain an IFT with a fixed 
number of states; by slightly modifying it, we can obtain an IFT generating 
any given recursively enumerable language (at the first step of the work of 
our IFT we introduce the code of the particular grammar which is used by 
the universal one in order to simulate the particular grammar; then, our IFT 
works as the universal grammar, hence it generates the language of the par- 
ticular grammar). This argument only shows that the hierarchy IFTn, n > 1, 
collapses, without any indication on the hight of this hierarchy. 

The hierarchy on the number of states collapses also in the case of iterated 
gsm’s in the form considered in [5], [9], [11]: the previous argument holds for 
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this case as well. In fact, the result is stated explicitly in [9] (Theorem 2.12), 
but the argument is much more complex: one first gives a normal form for 
iterated gsm’s, one proves that state-complexity families with respect to such 
normal form iterated gsm’s are full AFL’s, and then one uses the fact that 
the family RE is a principal AFL. No estimation of the number of levels of 
the obtained hierarchy is given in [9]. (The question is not considered in [5], 
[ 11 ]-) 

Lemma 5. RE C IFT^. 

Proof. Consider a language L £ RE,L C T*, and take a grammar G = 
(AT, T, 5, P) in the Geffert normal form generating it, [2]. That is, we have 
N = {S, A, B, C} and the rules in P are of the following forms: 5 -> x, for 
X G (iV U T)+, and ABC -> A. 

Consider three new symbols, ao,ai,a 2 , and construct the IFT 



7 = U{ao,ai,a2},5o,ao,{5i},P'), 

P' = {sqUo -> aiSa2So} U {soa -> aso \ a £ N UT U { 01 , 02 }} 

U — y xsq I S — y X £ P} U {sqA — > saB — y sb^) sbC — y sq} 

U {sqOi — y 5i} U {sio — y clsi | o £ T} U {5102 — y 5i}. 

At the first step we introduce the axiom of G, marked to the left with the 
symbol oi and to the right with the symbol 02 . At the subsequent transla- 
tions, the context-free rules 5 x of P are simulated by using the state sq. 
The deletion of a substring ABC is performed as follows: 

S0^iAPGX 2 h* XiSoAPGx 2 b X\SaBCx2 b X\SbCx2 F X\SqX2 F* XiX25q. 

Of course, none or several applications of rules in P, irrespective whether or 
not they are context-free or non-context-free, can be simulated at the same 
translation, but this does not change the generated language. 

We can reach the final state si only by erasing the symbol ai by a step 
soaiw h siW] after that, only terminal symbols can be parsed, hence the 
string w obtained in this way is a terminal one. After erasing the symbol ai, 
the final state cannot be reached once again, hence new rules in P cannot be 
simulated in a successful way. 

The presence of the symbol 02 ensures the fact that we do not reach the 
end of a string in a state sa or sb (note that 02 cannot be scanned in states 
sa,sb)- 

Consequently, L{j) = L(G), that is RE C IFT 4 . □ 

We do not know whether or not the result above is optimal. Anyway, 
IFT’s with three states are very powerful: 

Lemma 6. The membership problem with respect to IFT’s with three states 
is not decidable. 
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Proof. Consider two n-tuples of non-empty strings, x = (xi, . . . ,Xn),y = 
(2/1 j • • • j 2 /n)j over the alphabet {a, 6}. Let be two new symbols associated 
with a, 6, respectively. Denote by h the coding defined by h{a) = a', h{b) = b\ 
and by mi{x) the mirror image of x, for any string x. We construct the IFT 

T ({^Oj j ^2} 5 6 , G , 6 , QjQj c, (ij- , sqj ^Oj {^0} j -^)j 

with the rules: 



1. soao -> cxid mi{h{yi))cso, for 1 < ^ < n, 

2. soa asoj for a E {a, 6, a',6',c}, 

3 . sqc? — Xid mi{h{yi))so, for 1 < i < n, 

4. Sod So, 

5. 5qG — ^ 5i, > Sq, 

6. ->* 52, S 2 b‘ “> 5 q. 

It is easy to see that after replacing ao, at the first step, with a string 
cxid mi{h(yi))c, for some 1 < i < n, at the next iterations one replaces 
the occurrence of d by a string of the form xjd rni{h{yj)), for 1 < j < n. 
In this way, we obtain all strings of the form cxi^^xi^ ...xi^d mi{h(yi^)) 

. . •mi(h{yi^)) mi{h{yi^))c, for some r > 1,1 < ij < n, 1 < j < r. 

Then, after removing the symbol d, by the rule of type 4, the following 
iterations either leave the string unchanged (by using rules of type 2), or 
a pair aa' or 66' is deleted, via states 5i,S2, respectively (by using rules of 
types 5 and 6). If we arrive in si (after erasing an occurrence of the symbol 
a) and the next symbol is not a', then the work of 7 is blocked (at least the 
rightmost occurrence of the symbol c has remained non-scanned) . The same 
happens if we arrive in S2 (after deleting an occurrence of the symbol 6) and 
the next symbol is not 6'. 

Therefore, we can eraise all symbols in between the two occurrences of the 
marker c if and only if h~^ {mi{h{yi^)) . ..mi{h{yi^)) mi{h{yi^))) — mi{xi^Xi^ 
...Xi^). This means that Xi^Xi^ . . .Xi^ = h~^ {mi{mi{h{yi^)) • • • mi{h{yi^)) 
fni(h(pij))). Because mt(mi(h(pij) . ■ . mi(h(pij)mi(h(yij)) = h(yi,)h(pi^) 

■ ■ -fi(Pir) = HyiiVh ■ ■ - yi^)> h is one-to-one, this is equivalent to XiiXi^ 

■ ■ ■ ■ ■ - yi,., that is, to the fact that the Post Correspondence Prob- 

lem for the n-tuples x,y (abbreviated, PCP{x,y)) has a solution. Conse- 
quently, the string cc belongs to the language ^(7) if and only if PCP{x, y) 
has a solution, which is not decidable. □ 

Lemma 7. ETOL C IFT 3 . 

Proof. For every language L G ETQL there is an ETOL system which 
generates L and has only two tables. Let G = {V,T,w,Pi,P2) be such a 
system. Let oq, c be new symbols and construct the IFT: 

7 = ({so,si,S 2},TU {ao,c},so,oo, {si},P), 
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P = {soao cwso} 

U {sqC — c5o} U {soa -> xsq \ a x e Pi} 

U {sqc — > CS2} U {52a -> XS2 \ a X G P2} 

U { 5 qc Si} U {51a -> asi I a G T}. 

It is easy to see that L{G) = L{j). If one stays in state sq when scanning 
c, then one has to completely rewrite the string according to the rules in Pi 
(the passing to si or to S2 is possible only when one scans c); if one passes 
to 52 after scanning c, then one has to use the rules in P2. The rules in the 
two tables cannot be mixed. Finally, if one passes to 5i after scanning (and 
removing) c, then the work of 7 should be finished: even if the rules in Pi 
can be used again and again, the final state 5i cannot be reached any more. 
The use of 5i also ensures that the obtained string is terminal. □ 

Synthesizing the results above, and using the known relations among 
Chomsky and Lindenmayer families, we obtain: 

Theorem 3 . (i) IFTi = OL C EOL C IFT2 C /FT3 C IFT4 = RE. (ii) 
CF C IFT2, ETOL C IFT3. 

We do not know which of the non-proper inclusions in this theorem 
are proper. Also the relationships between the family CS and the families 
IFTz^IFTs are open. According to the next result, if CS C IFT2, then 
RE C IFTs. 

Theorem 4 . If CS C IFTn, for some n > 1 , then RE C /PTn+i. 

Proof. Consider a recursively enumerable language L CT"". Take a type -0 
grammar C = (N, T, 5 , P) generating L and a new symbol, c. We construct 
the length-increasing grammar C’ — {N, T U {c}, 5 , P'), with 

P' = {u V \ u ^ V G P,\u\ < li;!} 

U {u vc^ \ u V G Pj\u\ = |i;| H- 5 , 5 > 1 } 

U {ca -> ac, ac ca \ a G N U T}. 

It is easy to see that L{C) C Llll {c}* and that for each string w G 
L there is a string w‘ G L(G')^w' G {w} III {c}*". (We have denoted by 
lii the shuffle operation, defined hy x W 1 y = {xiyiX2V2 • ■ ■ Xnyn \ ^ = 
X1X2... Xn,y = 2/12/2... 2/n,n > l,Xi,2/i € V*,l < i < n}, for x,y G V*, 
and extended in the natural way to languages.) Indeed, the length-decreasing 
rules of P are replaced by monotonous rules which introduce occurrences of 
the symbol c; this symbol is moved freely to right and to left, making room 
for left hand sides of rules in P. 

Consider now an IFT 7 = (AT, V, 5 q, oq, P, P") generating the context- 
sensitive language L{C') and having card{K) = n. Let ao,ci,C2,C3 be new 
symbols and si a new state, and construct the IFT 

7 ' = {^'U{si},y U{ao,ci,C2,C3},so,ao,{si},-P"')> 
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where P'" contains all transition rules in P" as well as the following transi- 
tions: 

1. soaQ C1XC2S, for all 50^0 xs e P” with x £ V*,s e K, 

2. 5qCi CiSq, 

SC2 C2S, for all 5 € K, 

3. SC2 -> C35, for all 5 E F, 

4. 5oCi Si, SiC — > Si, S1C3 — > Si, 

sia asi, for all a £T. 

After replacing by a string introduced by 7 when scanning ao? bounded 
by the markers ci , C2, the IFT 7' simulates the work of 7, modulo the passing 
over Cl and C2 and leaving them unchanged. The symbol C3 cannot be scanned 
in a state of K. This symbol can be introduced only from a state in F, by 
a rule sc2 -> C3S,s G F. If such a rule is used, then the obtained string, of 
the form ciwcz^ cannot be further translated by the rules in P” . This means 
that a rule of the form sc2 C3S,s G F, is used only once, at the end of 
simulating in 7' a sequence of translations in 7. 

On the other hand, the symbol C2 cannot be scanned in the state si , hence 
the work of 7' cannot be finished as long as the symbol C2 is present in the 
string. Thus, a rule sc2 -> C3S, 5 G F, must be used. This ensures the fact 
that the work of 7 ends in a final state, as imposed by F (note that the states 
in F are no longer final states in 7'). 

A string of the form ciwcs can be processed on the path from sq to si] 
all symbols ci,C3,c are erased, all symbols in T are left unchanged. This is 
the only way of reaching the end of a string in the final state si. Therefore, 
if w above is a string from L{G') such that w G {z} lii {c}*, for z G L, then 
the string generated by 7' is exactly In conclusion, L(j') = L, that is, 
L G /FTn+i. □ 

We close this section by mentioning some closure properties of families in 
the hierarchy IFTn,n > 1; proofs can be found in [3]. 

Theorem 5. If L,Li,L2 G IFTn and h is a morphism, then h(L), 

Li U L2, and L1L2 are in IFTn-\-i,n > 1. 

5 Back to Computing by Carving 

The main result of this paper is the fact that non-deterministic iterated finite 
state sequential transducers with four states characterize the recursively enu- 
merable languages. This implies that the number of states does not induce 
an infinite hierarchy of languages which are C-REG computable: in view of 
Theorem 2.1, a language is C-REG computable if and only if it is the com- 
plement of a language in RE = IFT4. Thus, if we take into account the 
number of states of an IFT generating the complement of a C-REG com- 
putable language, then we can have at most four levels in the hierarchy of 
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C-REG computable languages. If we take into account the number of states 
of the gsm g describing a regular sequence of languages, identified by a pair 
then the hierarchy is still lower: there are at most three levels. To 
this aim, we will repeat the proof from [6] of one implication in Theorem 2.1, 
making use of the bctsic idea used here in the proof of Lemma 4.5. 

Let us denote by CREGn,n > 1, the family of languages of the form 
M - for M, Li regular languages and g a gsm with at most n states; 

by CREG we denote the union of all these families, that is, the family of all 
C-REG computable languages. 

Theorem 6 . CREGi C CREG2 C CREGs = CREG. 

Proof. The inclusions C are obvious. According to Theorem 2.1, each 
language L C T*, L € CREG, has the complement in RE. Take such a 
language L and consider a grammar G = (N, T, S, P) for the language T"" — L\ 
take G in the Geffert normal form, that is with N ~ {5, A,B,C} and with P 
containing context-free rules S x,x £ {N U T)*, and the non-context-free 
rule ABC A. 

We construct the regular sequence of languages starting with Li = {S} 
and using the gsm 

g = {{sq,sa,sb},N UT,NUT,so,{so},R), 

R — {sqQ^ OCSq \ (X £ N \JT} U {5o5 XSq I X £ P} 

U {sqA SA, SaB — > SB, SbC So}^ 

It is easy to see that g*{Li) n T* = L(G) (at each iteration of g one can 
simulate the application of a rule in P\ if several rules are simulated at the 
same iteration, then this does not change the generated language). 

Therefore, for the regular language M = T* we obtain M — g’^iLi) = 
T* — L(G) = T* — (T* — L) = L. In conclusion, L £ CREGs. 

The inclusion CREGi C CREG2 is clearly proper, because by iterating 
a gsm with only one state we get a language in IFTi = OL. □ 

The proof before does not imply that RE = IFTs , because one state has 
been saved here by using the language M, of a specific form, which makes 
unnecessary the use of a special (final) state in order to check whether or not 
the string produced by iterating the gsm is terminal. 

The previous theorem shows that in order to classify the C-REG com- 
putable languages we need other parameters describing the complexity of 
the regular sequences of languages, the number of states is not sufficiently 
sensitive. Such more sensitive parameters remain to be found. 

The case of deterministic iterated transducers remains to be investigated. 



Note. We are much indebted to H. Fernau for useful remarks on a pre- 
vious version of this paper. 
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Summary. Some first steps are proposed for a general approach to the structure 
and typology of formal languages in respect to the length of their words. A special 
attention is paid to the asymptotic behavior of various parameters involved in this 
problem. 



1 Introduction 

Given a language L, how many words of a certain length are in L? Related 
to this question, some interesting concepts and problems were considered in 
formal language theory and perhaps among the first results was the famous 
pumping lemma due to Bar-Hillel, Perles and Shamir [1] and having as one 
of its consequences the fact that the length set of an infinite context-free 
language contains an infinite arithmetic progression. 

Let us denote by fi (n) the number of words in L whose length is equal to 
n; this is the density function associated with L. Various (explicit or implicit) 
results related to this function, mainly in the particular case when L is regular 
or context-free, are due to Schiitzenberger [12], Chomsky and Schiitzenberger 
[3], Eilenberg [4], Salomaa and Soittola [11], Berstel and Reutenauer [2]. The 
“generating function” associated with a languages L under the form of an 
infinite series giiz) = Y2n>o where an — fiin), opened a new road by 
the theorem asserting that, when L is an unambiguous context-free language, 
the analytic function giiz) is an algebraic function over the field of rationals 
(Chomsky and Schiitzenberger [3]). However, this very attractive approach 
failed to lead to a general theory of length sets in the field of formal languages. 

Our aim is to stimulate the development of a general framework in the 
study of the length of words and to accomplish some first steps in this direc- 
tion. 



2 A Characterization of the Density Function 

Given an arbitrary infinite sequence (an) of positive integers (zero included), 
under what conditions does there exist a language L over the alphabet A, 
whose density function is given just by fiin) ~ an^ Denoting by p the 
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number of elements in A, it is easy to see that a necessary condition for the 
existence of L is On < for n = 1,2,... Is this condition also sufficient 
for the existence of LI The answer is affirmative, because we can choose in 
various ways an words of length equal to n among the total number of 
such words on A (for n — 1,2,...) and define L as the language containing 
the words selected in this way. We conclude: 

Proposition 1. A necessary and sufficient condition for an infinite sequence 
{an) of positive integers to be the density function of a language L on the 
alphabet A {i.e., fii'o) = OLn for n = 1,2,...) is an < p^, where p is the 
cardinality of A. 

We also have: 

Proposition 2. If card A = \, then the language L in Proposition 1 is 
uniquely determined. If card A> then there are infinitely many languages 
on A having the same given density function. 

Remark 1. Proposition 2 suggests the consideration of the following equiv- 
alence relation: two languages L and M are equivalent in length if /l(^) 
fM{n) forn= 1,2, ... 

We may ask: how much different can be two languages equivalent in length! 
The question is relevant only when card A> \ and in this case it is easy to see 
that there always exist disjoint languages equivalent in length. For instance, 
if A = {a,b}, then L = {a^ | n = 1,2,...} and M = {5^ | n = 1, 2, . . .} 
are equivalent in length, because /l(^) = /m(^) = 1 for n = 1, 2, . . . We 
can also take L = {a, | n = 1,2,...,/? = 1,2,...}, M — {b,b^a^ | 
n = 1,2,...,/? = 1,2,...}. L and M are disjoint and equivalent in length: 
/l(1) = /m(1) = Ij because a in L and 6 in M are the only words of length 
one; /l(2) = /m(2) = 1, because ab in L and ba in M are the only words of 
length 2; /l(3) = /m(3) = 2, because ab‘^,a‘^b in L and ba^,b‘^a in M are the 
only words of length 3, etc. This example shows that we need a more general 
equivalence relation: two languages L and M are said to be almost- equivalent 
in length if there exists a positive integer p such that fi{^) = fM{^o) for 
any n> p. An interesting problem is to determine what classes of languages 
(for instance, those in Chomsky hierarchy) are invariant through almost- 
equi valence in length. 

So far, we paid no attention to the generative nature of the language L. 
In view of the pumping lemma [1], we can bring the following information: 

A necessary condition for the language L in Proposition 2 to be context- 
free is that the sequence {an) in Proposition 2 contains an infinite arithmetic 
progression. 

It would be interesting to obtain more information about the behavior 
of the density function for various classes of languages. The particular case 
of regular languages was investigated by Szilard, Yu, Zhang and Shallit [13], 
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who proved the existence of some gaps for their possible density functions. 
For instance, there is no regular language that has a density of the order 
y/n, nlogn, or 2"^; see, for more information, Yu [14]. 

3 Full Intervals and Lacunary Intervals 

Given a language L on an alphabet A, a full interval for L is a sequence of 
consecutive positive integers, such that: 1) for each term k in the sequence 
there exists a word in L whose length is equal to k; 2) if z is the first term 
in the sequence, then there is no word in L whose length is equal to z — 1; 
3) if j is the last term in the sequence, then there exists no word in L whose 
length is equal to j -h 1. We will denote this interval by [z, j]. The length of 
this interval is the number of positive integers k such that i < k < j. It may 
happen that i = j (intervals of length one), z = 0 (in this case condition 2 
above is devoid of object), or j = oo (in this case we should write [z, oo) and 
condition 3 above is devoid of object). 

Given two full intervals [i,j] and [p,r] for L, j < p, we say they are 
consecutive if there is no positive integer m, j < m < p, which is the length 
of a word in L. In this case, we say that the sequence j-1-1, j+2, . . . ,p— 2,p— 1 
is a lacunary interval for L and we denote it by [j 4* l,p — 1]. 

Proposition 3. Given two arbitrary sequences (nik) and (uk) of strictly pos- 
itive integers, there exists a language L on the alphabet {a} such that ruk is 
the length of the k-th full interval of L, while Uk is the length of the k-th 
lacunary interval of L {k — 1,2,...). 

Proof. A language L with the required properties is {a^ | p G 5}, where S is 
the union of the following full intervals for L: [1, mi] , [mi 4- ni 4- 1, mi 4- ni 4- 
^2], [mi4-ni4-m2 4-n24-l,mi4-ni4-m2 4-n2 4-m3],. . . , [mi4-ni4-m2-hn2 4-. . .4- 
mk+nk-\-l,mi-{-ni 4-m2 4-n2 4-. . .-\-mk-^nk+mk^i], . . . , while the lacunary 
intervals for L are: [mi 4- l,mi 4-ni], [mi +rzi 4- m2 4- l,mi 4-ni 4-m2 4-n2], 
[mi 4- ni 4- m2 4- ri2 4- m3 4- l,mi 4- ni 4- m2 4- ri2 4- m3 4* n3] , ... 

Remark 2. Proposition 3 remains valid when the alphabet {a} is replaced by 
an arbitrary finite non-empty alphabet A. In order to avoid the trivial case 
when some letters in A are not used in the considered language on A, we 
define complete words on A as those words that use all letters in A. 

Proposition 4. Proposition 3 remains valid when {a} is replaced by an ar- 
bitrary finite non-empty alphabet. 

Proof The non-trivial case refers to languages whose words are complete. Let 
us proceed by induction. The first step was accomplished by Proposition 3. 
For the second step, let us suppose that Proposition 3 is valid for an alphabet 
An-i of cardinality 1. This means that given the sequences (nik) and (uk) 
of strictly positive integers, there exists a language L^-i such that ruk is the 
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length of the ifc-th full interval of Ln-i, while rik is the length of the A:-th 
lacunary interval of Ln-i* Consider the A;-th full interval [i - 1, j - 1] for 
The language Ln we are looking for will be obtained by adding one 
more letter to the words in L„_i, so the length of each word in Ln will 
increase with one in respect to words in Ln-i> The interval [i — 1, j — 1] will 
become the full interval [i,j] for L^, the length of [i,j] being the same as the 
length of [z - 1, j - 1], i.e., rrik- A similar phenomenon will occur with the 
lacunary intervals. 



4 Asymptotic Behavior 



Let us consider, for any language L, the ratio /L(zi)/n, where is the 
density function of L. A relevant parameter of the asymptotic behavior of 
L in respect to the length of its words will be the behavior of the above 
ratio, when n is increasing. Since /l(^) < (see Proposition 1), we have 
0 < < ^5 where p = card A, so limn-^oo(p”/^) = oo- Obviously, we 

will have some times limn->oo(/L(^)/^) = 0, for instance, when /l(zz) < k, 
for a given constant k; the language L is called in this case slender, see Paun 
and Salomaa [8], [9], [10]. On the other hand, we may have /l(zi) = p^ for 
all n enough large (when we introduce in L all words of length n, i.e., L is 
almost universal), so limn-^oo(/L(zz)/n) = oo. 

More generally, we can assert: 

Proposition 5. Given 0 < a < oo, there exists a language L with 



n->oo ^ 



= a. 



Proof. The cases a = 0 and a = oo were already settled, so let us suppose 
0 < a < 00 . If a is rational, a — m/r {m,r positive integers such that 
m/r is irreducible), then we consider an alphabet A of cardinality p > 2 
and we choose L over A such that there are km words of length kr in L, 
for k = 1,2,... Thus, fL{kr)/kr = kmjkr = m/r for any positive integer 
k, while /l(zi) = 0 for any n which is not a multiple of r. This is possible 
because, for k enough large, km < indeed, m < {p^^)/k for k enough 
large, due to the fact that lim^b-^oo (p^^)/^ = oo. 

If a is irrational, then there exists an increasing sequence of rational num- 
bers (pn/Qn) converging to a. We may assume that (g^n) is strictly increasing 
(so (pn) is increasing, too). Define L by the following rule: for each n, L con- 
tains Pn words of length equal to qn- We have /^(^n) = Pn for n = 1, 2, . . . It 
follows that 



hiqn) 

qn 



Pn 

qn 



1 . hiqn) 
so lim — ^ — - = 

n^oo q^ 



For any n wich is not a term of (qn) we put /^(n) = 0, so 



lim sup Mi) 

n-4oo 



= a. 



n 
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Remark 3. The converse of Proposition 5 is also true: to any language L we 
can associate a non-negative number a(L) such that lim sup(/l (n) /n) = a(L) 
when n ->• 00 . We can call L rational (irrational, algebraic, transcendental) 
if a(L) is rational (irrational, algebraic, transcendental, respectively), and 
it would be interesting how is this typology of a(L) represented for various 
classes of languages. 

5 A Generalization and a Typology of Slender 
Languages 

The considerations above offer a new framework to look at a slender language 
L, defined by the existence of a natural number k such that flip) < k for 
n = 1,2,..., [8], [9], [10], [6], [7]. Obviously, for any slender language L 
the associated parameter a{L) (see Remark 3) is equal to zero. A natural 
question arises: is the converse of this assertion true? The answer is negative, 
as it follows from 

Proposition 6. There exists on an alphabet A with card A > 1 a language 
L which is not slender, hut 



lim 

n—^oo 



hjn) 

n 



= 0 . 



Proof. Define a sequence (an) by On = the number of prime numbers not 
larger than n. In view of the fundamental theorem of prime numbers, there 
exists A: > 1 such that an < k{n/ Inn). On the other hand, for n enough large 
and because the cardinality p of A is strictly larger than 1, 



On < k{n/\nn) < p^, so an < p”- 



We can apply Proposition 1. It follows that there exists a language L on A 
such that, for n enough large, /l(^) = CLn- On the other hand, we have 



an 

n 



< k ■ 



n 



n(lnn) 

and the proposition is proved. 



- — , SO hm — = 0, 
Inn n-^oo 71 



Let us call a language L asymptotically zero if lirnn->oo(/L(^)/^) = 0. It 
follows from Proposition 6 that such languages are an effective generalization 
of slender languages. (Note that this extension is in a direction different from 
that considered by Honkala [5].) Moreover, in respect to the speed of the 
convergence to zero of the sequence (/L(n)/n) we get an infinite hierarchy 
of asymptotically zero languages. Slender languages are at the bottom of this 
hierarchy, because for these languages, for n enough large, all terms of the 
considered sequence are equal to zero. But slender languages, in their turn, are 
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disposed in an infinite hierarchy: we say that a slender language is of rank k 
if the number of the full intervals of the associated set | n = 1, 2, . . .} 

(this number is in any case finite for any slender language) is equal to k. 
There exist, for any strictly positive integer A:, some languages of rank equal 
to k and having all their k full intervals of finite length. These languages are 
obligatory slender. 

There is another hierarchy of slender languages, proposed by Paun and 
Salomaa: a language L is Ar-slender if /l(^) < k for any n = 1,2,... For 
every positive integer k there are fc-slender languages that are not {k — 1)- 
slender. Obviously, a A:-slender language may have different ranks. If we call 
a language essentially k-slender when it is /i;-slender, but not {k - l)-slender, 
then we can assert that the rank of a language tells us almost nothing about 
its essential slenderness. So, rank and essential slenderness are rather incom- 
parable. Perhaps some restrictions appear when we consider some specific 
classes of languages. 



6 Fullness Versus Lacunarity 



Propositions 3 and 4 suggest the introduction of some relevant parameters 
measuring the asymptotic behavior of the fullness-lacunarity ratio. A first 
idea is to measure this behavior by the parameter 



A = lim 

k-^oo Tlk 



When A = oo, the language is said to be rich in length. When A = 0, the 
language is poor in length. A language which is poor in length may not be 
slender, as it can be seen by taking mk = the largest positive integer not larger 
than y/uk (this is possible in view of Propositions 3 and 4) . Languages poor 
in length form an infinite hierarchy, in respect to the speed of convergence 
to zero of the ratio ruklnk- It would be interesting to relate this hierarchy 
to the hierarchy of asymptotically zero languages, the extension of slender 
languages considered in Section 5. 

Another, more relevant, parameter measuring the ratio between fullness 
and lacunarity is 



/i = lim 






n->oo Nn 



where = Yl'^k and iV„ = the sum being considered for all full 

intervals and for all lacunary intervals contained in the interval [0,n]. 

For both A and p it may happen that the respective limit does not exist; 
in such cases we replace it by lim sup. The language L will be considered 
balanced if there exists a strictly positive number K such that ^ ^ < K 

for any k enough large or, better, < ^ < K ioi any n enough large. L 
is predominantly full if A > 1 and predominantly lacunary if A < 1. If A = 1, 
then L is perfectly balanced. If mj^ = Cuk for any k enough large, then the 
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language is predominantly full when the constant C is > 1, predominantly 
lacunary if C < 1 and perfectly balanced if C = 1. 

Since, in view of Propositions 3 and 4, rrik and rik are arbitrary positive 
integers, A can be an arbitrary positive real number and it can also be infinite. 
We get a potential typology of languages, in respect to the nature of A. If 
the sequence {rrik/nk) is convergent, then we will say that the corresponding 
language is convergent in length] otherwise it is divergent. Divergence can 
be of two types: if the limit A exists, but it is infinite, then we say that the 
language is divergent to infinity] otherwise, it is oscillating and we direct our 
attention to the superior limit of {ruk/nk). Similar distinctions can be made 
in respect to the parameter /x. Shortly, we can assert that the typology of 
languages in respect to A and /x is reflected by the typology of real numbers. 
The language L will be considered scattered in length if all its full intervals in 
length are of length equal to one. If such a language is convergent in length, 
then it converges to a number between zero and one; conversely, any real 
number between zero and one (0 and 1 included) is the limit A associated 
with a suitable language scattered in length. 

An interesting situation occurs when A = 1. In this Ccise both L and its 
complementary are scattered in length. However, this is a very restrictive 
condition; it corresponds to one of the following two cases: 1) L contains all 
words whose length is an odd number; 2) L contains all words whose length 
is an even number. 

7 Pairs of Languages and the Fullness-Lacunarity 
Duality 

Two languages L and M are said to be heterogeneous in length if the length 
of a word in L (M) is never the length of a word in M (L, respectively). For 
instance, L = {a^b^ | n = 1,2, . . .} and M = {a'^b^ | m 4* n odd and prime} 
are in this situation, because any word in L has as its length an even integer, 
while any word in M has as its length an odd integer. Denoting by s{L) the 
length set of L, for an arbitrary language L, heterogeneity in length means 
s(L) n s(M) = 0. If s{L) = s(M), then L and M are weakly equivalent in 
length. Obviously, equivalence in length implies weakly equivalence in lengthy 
hut the converse is not true (excepting the case of an alphabet with only 
one element). For instance, if we take A = {a, 6}, the universal language A* 
and the language {a'^b^ \ > 0} are not equivalent in length, but they 

are weakly equivalent in length. Weak equivalence raises problems similar to 
those related to equivalence in length. The main question is: how different can 
be two languages weakly equivalent in length? Taking into consideration that 
we are mainly interested in the behavior of words with an increasing length, 
two more equivalence relations have to be considered: almost equivalence in 
length and almost weakly equivalence in length. L and M are almost equivalent 
in length if there exists a positive integer p such that = fM{n) for any 
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n > p, L and M are almost weakly equivalent in length if the symmetric 
difference s{L)As{M) is finite. 

L is poorer in length than M if s{L) C s{M); equivalently, M is richer 
in length than L. It may also happen that L and M are not comparable, 
without being disjoint. The “almost” variant of “disjoint”, “poorer in length 
than” is obtained when some suitable finite subsets of s{L) and s{M) have to 
be ignored in order to obtain the respective relations. Examples of all these 
situations can be easily obtained. 

Two languages L and M are said to be complementary in length if they 
are heterogeneous in length and if s{L) U s{M) = {0, 1, 2, . . .}. 

Proposition 7. If card A > 1, then complementarity and complementarity 
in length are not comparable; for card A = 1, they are equivalent. 

Proof. Let L = {a^b^ | n = 0,1,2,...} and M = {d^b^ | m odd, n even, 
n,m> 0}. We have s{L) = {0, 2,4, ... , 2n, . . .} and s{M) = {1, 3,5, . . . , 2n + 
1,. . .}, so s{L) U s{M) = the set of positive integers, while L and M are 
heterogeneous in length. However M is only a part of the complementary 
set of L; so, complementarity in length does not imply complementarity. 
Conversely, let L be the same as above and M = {a^b'^ \m ^n, m, n > 0}. 
L and M are complementary, but not complementary in length. The second 
part follows firom Proposition 2. 

We will now give a characterization of languages heterogeneous in length 
and of languages complementary in length. 

Proposition 8. L and M are heterogeneous in length if and only if any 
interval full in length for L (M) is lacunary in length for M {L, respectively) . 
L and M are complementary in length if and only if any interval lacunary 
in length for L (M) is full in length for M (L, respectively) . 

Proof Suppose that L and M are heterogeneous in length, i.e., s{L)C\s{M) — 
0 and let I be an interval full in length for L. In order to prove that / is a 
lacunary interval for M, let us accept, by contradiction, the existence of a E 
lC]s{M). From a E / and I C s(L) it follows that a E s(L), so a E s{L)ns{M), 
in contradiction with the hypothesis. The validity of the statement obtained 
by changing L with M and M with L follows from symmetry considerations 
(heterogeneity is a symmetric relation): if the hypothesis is symmetric in 
respect to L and M, then the conclusion should be symmetric too. 

Conversely, if any interval full for L (M) is lacunary for M (L, respec- 
tively), then any a in s{L) belongs to a full interval / for L, which should 
be, by hypothesis, lacunary for M, so a cannot be in s(M); this means that 
s{L) n s{M) = 0, so L and M are heterogeneous in length. 

Suppose now that L and M are complementary in length; this means that 
s{L) n s{M) = 0 and, moreover, that s{L) U s{M) - {0, 1,2, . . .}. If 7 is a 
lacunary interval for L and a E 7, then a is absent from s{L), so, in view of 
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the complementarity in length, it follows a € s(M). so / C s(M) and I is full 
in length for M. For symmetry considerations, things behave similarly when 
L is replaced by M and M by L. 

The use of the operator s{L) requires the explicit status of its behavior, 
as it is given by 

Proposition 9. For any pair of languages L,M we have: a) s{L U M) = 
s(L) U s(M); b) s{L fl M) C s{L) H s{M); c) If L C M, then s{L) C s(M), 
but the converse is not true; d) s{LM) = s{L) H- s(M), 

Proof If a € s(LU M), then there exists x £ LUM with a = s{x). x £ L, 
then a £ s{L)] if a: € M, then a £ s(M); in both cases a £ s{L) U s{M). 
Starting now with the last relation as hypothesis, we have a £ s{L) or a € 
s(M), but in both cases a £ s(L U M) and a) is proved. 

If a G s(L D M), then a £ s(L) and a £ s{M), so a E s(L) D s{M). The 
converse is not true, as it can be seen when L and M are complementary, 
but not complementary in length, a possibility assured by Proposition 7. So, 
b) is proved. If L C M, then from a £ s{L) follows the existence of x £ L 
with s{x) — a and, because x £ M too, we have a £ s{M). The converse 
is not true, as it can be seen by taking L = | n > I}, M = {a^b^ | 

m ^ n, m, n > o}. Indeed, s{L) = {2n | n = 1, 2, . . .}, 5(M) = {1, 2, . . .}, so 
s{L) C s{M), but L is not contained in M; moreover, L D M = 0. So, c) is 
proved. 

Let us now take a £ s{LM). This means the existence of u £ L^v £ M 
with a = s{uv). But s(uv) = s{u) -f s(v), so a £ s{L) -h s{M). Conversely, if 
a £ s{L) -f s(M), then there exist u £ L and v £ M with a = s{u) + s(v) = 
s{uv), where uv £ LM^ so a E s{LM) and d) is proved. 

All the above concepts and properties can be extended to classes of lan- 
guages. Given a class F of languages and a binary relation r between lan- 
guages, we may say that the class F satisfies the relation r if for any two 
languages L, M in F we have L r M. The relation r can mean here hetero- 
geneity, almost heterogeneity, complementarity in length, etc. 
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Robert McNaughton^ 



Summary. This review paper will report on some recent discoveries in the area 
of Formal Languages, chiefly by F. Otto, G. Buntrock and G. Niemann. These 
discoveries have pointed out certain break-throughs connected with the concept of 
growing context-sensitive languages, which originated in the 1980’s with a paper 
by E. Dahlhaus and M.K. Warmuth. One important result is that the deterministic 
growing context-sensitive languages turn out to be identical to an interesting family 
of formal languages definable in a certain way by confluent reduction systems. 



1 Growing context-sensitive languages 

There are several reasons for proposing that the family of GrCSL’s (growing 
context-sensitive languages) be considered as a new level in the Chomsky 
hierarchy of languages. The insertion would be between the CFL’s (context- 
free languages) and the CSL’s (context-sensitive languages), in effect making 
a new family designated as the “type one-and-a-half languages” . This fam- 
ily began to receive attention in 1986, when Dahlhaus and Warmuth pub- 
lished their result [5] that the complexity of its membership problem had a 
polynomial-time algorithm, in contrast to the P-space-complete membership 
problem for the larger family of context-sensitive languages. 

A GrCSG (growing context-sensitive grammar) is one in which \a\ < \l3\ 
for every rule a ^ /3] a, GrCSL is the language of a GrCSG. This class 
of grammars is a proper subclass of the class of CSG’s (context-sensitive 
grammars) where rules are also permitted in which \a\ = \/3\. (See pp. 200- 
214 of Arto Salomaa’s 1969 book [3] for an elucidating treatment of context 
sensitive grammars.) 

The GrCSL’s are just one of many studied families properly between the 
CFL’s and the CSL’s. Another was the family of CSL’s with linear-bounded 
derivations (see, e.g., Ron Book’s dissertation [2]), i.e., languages having a 
CSG with a bound B such that every w £ E* derived in the grammar has 
a derivation whose length is < B\w\. In 1964 Gladkij proved in [1] that the 
CSL 

{wcw^cw\w £ {a, 6}*} 

* Paljon onnea, Arto. In other words, Onnellista syntymapaivaa. 
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does not have linear-bounded derivations (see also the appendix to [2]). 
means w written backwards.) 

More recently it has been proved that the GrCSL’s are a proper subfamily 
of the family of CSL’s with linear bounded derivations (see [12], and also [13], 
Corollary 5.4). However, the latter family will no longer be of concern in this 
paper. 

From the work of Lautemann [6] and Buntrock [12] it follows that the 
language {ww\w C {a, 6}*} is not a GrCSL. (See also [13], especially the 
penultimate paragraph of Section 1.) Thus we have an improvement on the 
Gladkij language for a paradigm CSL that is not a GrCSL. 

All CFL’s consisting of words of length > 2 are GrCSL’s. The easy proof 
is by a constructive modification of the Chomsky normal form. But not all 
GrCSL’s are context-free, e.g, {6a^”|n > 1}, a GrCSG for which is 

S — y SK\bcLci 

baaK -> baaaa 
aK — > Kaa 

The concept GrCSG is based on the length of strings. It is convenient 
to allow as GrCSG’s the grammars that satisfy a variant of the definition 
based on the weighted length of strings. A weighting function (j> on the words 
over an alphabet E maps each word to an integer satisfying the following: 
(1) 4>{x) > 0 for all X £ E, (2) <^(A) = 0 (A is the null string) and (3) (l>(xy) = 
<l){x) + (f){y) for all words x and y. We define a grammar to be a GrCSG in 
the new sense if there is a weighting function cf) on the words over the total 
alphabet such that 0(a) < 0(^3) for every rule a 13. Following [8] and [11] 
we can prove that, if G is a GrCSG in the new sense then there is a G' that 
is a GrCSG in the original sense such that L{G') — L{G) D EEE*. 

Note that a context-free grammar in Chomsky normal form whose lan- 
guage does not have the null word is a GrCSG in the new sense: take 0(x) = 1 
for X a variable and 0(x) = 2 for x a terminal. 

It will be convenient to adopt this new definition of GrCSG for the re- 
mainder of this paper, yielding the slight change in the definition of GrCSL. 

As is well known, a language is context-sensitive if and only if it is rec- 
ognized by a nondeterministic LB A (linear bounded automaton). We can 
get a corresponding result for GrCSL’s by modifying this automaton to one 
whose tape decreases in its weighted length at every move. The best way to 
work out this idea precisely is to follow Buntrock and Otto in Section 3 of 
[13], stipulating an automaton with two pushdown tapes, representing the 
portions of the LBA tape to the left of the head and the right of the head, 
respectively. An elaborate weighting function is defined for configurations of 
the automaton, satisfying the condition that if one configuration is followed 
by another then the weight of the latter is less than the weight of the former. 
This weighting function is based on a weighting function of words over the 
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alphabet, but it is far too complicated to be described here. (One trick is to 
get the effect of weighing each character of E more heavily on one pushdown 
than on the other.) 

We thus have a nondeterministic shrinking two-pushdown automaton and 
the result that a language is a GrCSL if and only if it is recognized by 
such a device. The proof given by Buntrock and Otto is somewhat similar 
to the proof that a language is a CSL if and only if it is accepted by a 
nondeterministic LB A. 

The question naturally arises as to which CSL’s are GrCSL’s and which 
are not. No broad answer has been given to this question. Some well known 
CSG’s using length-preserving rules have languages that turn out to be 
GrCSL’s, an example being the grammar: 

5 SABC\dABC 



BA AB 


CB-^BC 


CA-^AC 


dA — ^ dxx 


Q,A — dQ 


aB ab 


bB -4 bb 


bC-^bc 


cC cc 



Its language {da^b^c^\n >1} also has the GrCSG: 

S SK\SL\dabc 

cK Kcc bK -> Kbb 

aK Kaa daK daa 

cL Mccc cM -> Mcc bM — > Nbbb 

bN Nbb aN Kaaa daN daaa 

The weighting function is <p{x) = 1 if x is a variable, <^(x) = 2 if x is a 
terminal. (To derive da^'^b^^c^‘^ in this grammar we would begin by deriving 
dabcK^. But to derive da^^b^^c^^ we would note that 37 = 2^ -f 2‘^ -h 2^, and 
accordingly we would begin by deriving dabcKKLKL.) 

With a bit more trouble we could get a GrCSG for {da'^b'^c^} in the 
original sense, i.e., one in which the weighting function is (f){x) = 1 for all 
variables and terminals x. Also, if we wished to get rid of the d, which acts 
as a left-end marker, we could do so at the expense of further complication 
in the grammar. 

GrCSG’s have the advantage over other CSG’s in that, in each such gram- 
mar, the length of a derivation has an upper bound that is linear in the length 
of the word derived. Moreover, as mentioned, the membership problem for 
every GrCSL has a polynomial-time algorithm. (The proof in [5] goes over to 
our new definition of GrCSL.) 

Dahlhaus and Warmuth [5] prove that every GrCSL is log-tape reducible 
to some CFL. Buntrock and Otto [13] improve on this result by showing that 




An Insertion into the Chomsky Hierarchy? 207 



this reduction can be done as a one-way log-space reduction; that is to say, 
the GrCSL is (quoting Section 1 of [13]) “accepted by an auxiliary pushdown 
automaton with logarithmic space bound and polynomial time bound that 
uses its input tape in a one-way fashion.” 

As pointed out in [8], the family of GrCSL’s is an abstract family of 
languages, that is to say, this family is closed under union, concatenation, star 
iteration, intersection with regular languages, null-word-free homomorphisms 
and inverse homomorphisms. 

A persistent problem for theoretical computer scientists has been to find 
a family of formal languages that will include all programming languages or 
most programming languages. The family must be reasonably simple concep- 
tually and must not be so broad as to include languages that have properties 
that no reasonable programming language could have. This objective is nec- 
essarily vague and this is no place to attempt to refine it or even to discuss 
it, except to make a brief negative point about the family of GrCSL’s: If 
the family of formal languages must include a language of programs whose 
variables occurring in executable statements must also occur in declaration 
statements, and if the family includes variables of unlimited length, then the 
family of GrCSL’s is not a suitable family for this purpose. The argument for 
this negative assertion is based on the piece of evidence that {ww\w E {a, b}*} 
is not a GrCSL, which indicates that any programming language in which a 
defined program may have arbitrarily long variables, but only those that are 
declared, is probably not a GrCSL. 

The application of ideas from Theoretical Computer Science to actual 
computing is difficult to predict with any precision. But it helps to have some 
general idea of the possibility of some application, even if the exact nature of 
this application is vague. There will be more to say about the applicability 
of these ideas on GrCSL’s when we investigate the deterministic variety of 
them in the next section. 

2 The deterministic variety 

A GrCSL is deterministic if it is recognized by a deterministic shrinking two- 
pushdown automaton. The name for this automaton is rather long; let us call 
it a “D-shrink” for short. As it is an important concept, it deserves a formal 
definition. The following is adapted from the paper by Buntrock and Otto 
[13]: 

A D-shrink is a 7-tuple 



{Q,S,r,5,qo,B,F). 

Here Q is the set of states, S the input alphabet, F the tape alphabet {S C F 
and r n <5 = 0)) 9o is the initial state, B is the bottom marker of the two 
pushdown stores {B G F - E), F is the set of accepting states, and 5 is the 
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transition function: 



S:Qxr‘^ -^Qxr^xT U {0}. 

A configuration in a computation of a D-shrink is given as uqv where 
u,v £ r* and q £ Q. The idea is that u and v are the words on the left and 
right pushdown tapes, respectively. Normally B is the leftmost character of 
u and the rightmost character of v and occurs nowhere else. Where u = u'a^ 
V = bv', a,b £ r and S{q,a,b) = {q',wi,W 2 ), the word u'wiq'w 2 v' is the 
next configuration in that computation. If 6{q, a, b) = 0 then uqv is a halting 
configuration. An initial configuration is of the form Bq^vB where v is the 
input. The bottom marker B is never created or destroyed in a computation 
but may be sensed. Acceptance of v is either by final state (a configuration 
u'qv' where q £ F) or by empty store (a configuration BqB for q £ Q). The 
language of a D-shrink is the set of all accepted inputs. 

What makes the D-shrink shrinking is the stipulation that there is a 
weighting function 0 on strings over the alphabet Q U F such that, for 
S{q,a,b) = {q,wi,W 2 ), the condition (f>{wiqw 2 ) < (l>{aqb) holds. 

As mentioned in Section 1, a language is a GrCSL if and only if it i^ 
the language of a nondeterministic shrinking two-pushdown automaton ([13], 
Section 3). If a GrCSL is accepted by a D-shrink (i.e., a deterministic automa- 
ton of the variety) then the language is said to be a DGrCSL (deterministic 
growing context-sensitive language). I shall argue in the remainder of this pa- 
per that the family of DGrCSL’s is an important family of languages, perhaps 
more important than the larger family of all GrCSL ’s. 

The language {ba^ \n > 1} is a DGrCSL. One can design a D-shrink for 
it based on the grammar from Section 1: 

S — > SK\baa 

baaK -> baaaa 
aK -> Kaa 

Any word in the language of this grammar will be processed by the D-shrink 
according to a rightmost derivation in the grammar, which means that the 
word is processed fi:om left to right. For example, the rightmost derivation 
of the word has the line ba^KaaKK, to which the rule aK Kaa is 
applied to the rightmost aK, resulting in the line ba^KaKaaK. The auto- 
maton does things in reverse of the order in the derivation; for that step it 
might have Bba^Ka on the left tape and KB on the right tape, and might 
be in a state showing that Kaa is between the ba^Ka and the K. It then 
would push aK onto the left tape, and would go into a state showing that 
the null word is between the Bba^KaaK on the left tape and the KB on the 
right tape. 

We shall not verify that this automaton can be made deterministic, and 
therefore that the language {6a^"|n > 1} is a DGrCSL. The language 
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{da^b^c^\n > 1} is also a DGrCSL; details beyond the discussion of this 
language in Section 1 are omitted. 

It is not difficult to prove that the family of DGrCSL’s is closed under 
complementation. This observation enables us to prove the existence of CFL’s 
that are not DGrCSL’s. Such a language is {a, 6}* — {ww\w € {a, b}*}. If this 
CFL were a DGrCSL then its complement {ww\w G {a, 6}*} would also be a 
DGrCSL, and hence would be a GrCSL, which (as mentioned in Section 1) 
it is not. 

And so, although all null-word-free CFL’s are GrCSL’s, they are not all 
DGrCSL’s. This may be an unpleasantness that might dissuade some theo- 
reticians from accepting the family of GrCSL’s as a member of the Chomsky 
hierarchy. As it now stands each family in the hierarchy is a subclass of the 
deterministic subclciss of the family at the next level. (Incidentally, as noted 
in [12] and [13], all null- word-free deterministic CFL’s are DGrCSL’s.) 

Whether or not it deserves a place in the Chomsky hierarchy, the family of 
GrCSL’s is an important family. Indeed there is reason to regard the subclass 
of DGrCSL’s as being more important than the larger family of GrCSL’s. As 
will be shown in the next section, the DGrCSL’s can be characterized in 
terms of confluent rewriting systems, which gives them perhaps even more 
signiflcance. 

(Before going on to Section 3, let us pause to observe that the family of 
DGrCSL’s is not closed under union or intersection. The simple argument 
for intersection [14] is as follows: It is easy to see that the Gladkij language 
{wcw^cw\w G {a, 6}*} is equal to the intersection of two deterministic CFL’s, 
which are therefore both DGrCSL’s; but the Gladkij language itself is not 
a DGrCSL. That this family is also not closed under union follows by the 
DeMorgan law, since the family is closed under complementation. Other such 
results can be found in Section 5 of [14].) 

3 Confluent string rewriting systems 

Perhaps the greatest selling point for the family of GrCSL’s is its link with 
the theory of rewriting systems as it has been developing since 1970. More 
speciflcally, the selling point is for the family of DGrCSL’s, which turns out 
to be identical to a family of languages definable in a certain way by confluent 
string rewriting systems, as discovered recently by Niemann and Otto [14]. 

Briefly, a string rewriting system is a semi-Thue system. We focus on 
systems in which the application of a rule to a word results in a simplification 
of the word: for example, it may be that \/3\ < \a\ holds for each rule a 
/3. Such systems are often called reduction systems, since their purpose is 
to take a long word and gain some sort of understanding by reducing it 
to a shorter word. The length requirement is not a strict requirement, but 
one necessary property of a reduction system is that there be no infinite 
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derivations. Consequently every word can be reduced to an irreducible word 
(the Noetherian property). 

A further property that is desirable for a reduction system is that no word 
can be reduced to two distinct irreducible words (the confluence property). 
In reducing a word according to a confluent reduction system, it is sometimes 
possible to start reducing in two distinct ways at the same point in the word. 
But then the two reduction sequences must eventually come together. (For a 
good exposition of string rewriting systems, see the first two chapters of [9].) 

Some languages can be deflned by confluent reduction systems; if the al- 
phabet of the system is the same as the alphabet of the language then the 
language is a congruential language^ i.e., the union of some of the congruence 
classes of a congruence relation over E*. Unfortunately, most interesting for- 
mal languages are not congruential. 

However, if we allow ourselves to supplement the alphabet of the reduc- 
tion system to include some control characters along with the alphabet of the 
language, we get something that is more fruitful. The paper [7] investigated 
the question of which formal languages could be defined in this way. The re- 
sults and conceptual development of that paper have recently been surpassed 
in a remarkable way by Niemann and Otto [14], whom the present exposition 
will follow. 

Another desirable property that our reduction systems must have is that 
they be weight reducing in the sense defined in Section 1; that is to say, there is 
a weight function <f> such that, for every rule a (3, (f>{/3) < (t>{a). A reduction 
system has the generalized Church-Rosser property if it is confluent and 
weight reducing. (It has the strict Church-Rosser property if it is confluent 
and length-reducing. Except for a few isolated remarks, we shall generally 
ignore the strict property for the remainder of this paper in favor of the more 
general property.) 

A language L C E* is a GenCRL (generalized Church-Rosser language) if 
there exists a reduction system S with the generalized Church-Rosser prop- 
erty satisfying the following conditions: 

(1) The alphabet F of S contains Z* as a proper subset; 

(2) There are Y,ti and ^ 2 , where ti,t 2 € {F - E)*, Y e F - E and 

Y is irreducible in 5, such that, for all w £ E*, w £ L if and only if 

tiwt 2 Y (viz., Y is derivable from tiwt 2 in S) 

Notice that if we have a language L that is of interest to us then such a 
system S would be a nice thing to have for testing membership in L. Given 
any w £ E*, we would form the word tiwt 2 and reduce it modulo S. If the 
reduced word is Y then w £ L; if not then it is not. Since every rule of 5 is 
weight-reducing, the length of the reduction of tiwh is linear in \(t>(w)\, and 
hence linear in \w\. Each step of the reduction is rather easy; we simply scan 
the word that we have for a subword that is the left side of a rule. (Things 
can be done so that the total amount of time spent in scanning during the 
entire reduction is insignificant.) When we find such a subword we reduce the 
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word accordingly. If we find that there is no such subword and the word at 
that point is not simply Y then we know that the original word w is not in 
L. 

As proved in [13] and [14], a language L is a GenCRL if and only if it is 
a DGrCSL. In effect, the D-shrink is a suitable mechanism for reduction of 
the word in the reduction system; in fact, precisely suitable. This automaton 
is similar to the automaton conceived by Ron Book [4] to reduce a word 
according to a reduction system with the Church-Rosser property. 

In [14] it is demonstrated that every GenCRL is also a CRL; in other words 
the reduction system can be modified so as to allow the length function as 
the weighting function (i.e., for each x e F, (f>{x) = 1). In the same paper, it 
is also demonstrated that the reduction system can be modified so that, for 
every w G F*, tiwt 2 reduces either to Y (indicating “yes,” that it; € L), or to 
N (indicating “no,” that it is not); both A^and Y are in F — F. These results 
settled questions left open in [7]. Furthermore, they make the characterization 
of DGrCSL’s in terms of rewriting systems even more significant than they 
appear at first. They also strengthen our feeling that the DGrCSL’s constitute 
an important family of languages. 

An interesting open question concerns the language {ww^\w e {a, 6}*}, 
which is clearly a GrCSL, since it is context-free. I conjecture, however, that 
it is not a DGrCSL. In [7] there is a plausibility argument that it is not a 
CRL. 

In conclusion, I suspect there will probably be few theoreticians who will 
press for any modification of the Chomsky hierarchy. Nevertheless, I hope 
many will come to realize that both the family of GrCSL ’s and the family 
of DGrCSL’s will play important roles in the future of computer science. I 
am especially convinced of the importance of the DGrCSL’s, since they have 
such a solid link with the contemporary theory of rewriting systems, and, in 
particular, with string rewriting systems having the confiuence property. 
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Word Length Controlled DTOL Systems 
and Slender Languages 



Taishin Yasunobu Nishida 



Summary. We introduce a new controlled DTOL system, called a word length 
controlled DTOL system, or a wlcDTOL system for short. A wlcDTOL system is a 
DTOL system with a control function which maps from the set of nonnegative inte- 
gers to the set of tables. A wlcDTOL system derives exactly one word from a given 
word by iterating the table which is the value of the control function of the length 
of the given word. Thus a wlcDTOL system generates a sequence of words which 
starts from the axiom. We prove that every wlcPDTOL system generates a slender 
language. We also prove that there is a wlcDTOL language which is not slender. 
Since a slender language is applicable to cryptography, the family of wlcPDTOL 
languages may be useful for cryptography. 



1 Introduction 

The notion of slender languages and its applicability to cryptography is first 
introduced in [1]. A language L is slender if there exists a constant k such 
that L has at most k words of the same length. A slender language may be 
used as a key generation mechanism of a cryptosystem. Although the paper 
[1] mentions the Richelieu system, we prefer a system like the one time pad 
system since the former makes much longer ciphertext than the latter. As for 
the details of cryptography, see, for example, [3]. 

The cryptosystem uses slender language is as follows. The sender and 
receiver have the same slender language L. The sender selects a key from L 
which has the same length with the plaintext and enciphers the plaintext by 
adding letter to letter with the key. The receiver first selects candidates of keys 
which are included in L and have the same length with the ciphertext. Then 
the plaintext is recovered by subtracting one of the keys from the ciphertext. 
We note that the addition and the subtraction are performed under modulo 
n where n is the cardinality of the alphabet of plaintexts. 

The above cryptosystem will be strong and easy to handle if there is a 
family of slender languages which seem as if sets of random sequences and 
are generated in polynomial time. The necessary and sufficient conditions 
for a context-free language to be slender are known [2,7,8]. But context-free 
slender languages have simple forms and are easily inferred [12]. For the OL 
languages, a subfamily in which the slenderness problem is decidable is known 
[4-6]. The known slender OL languages also have simple repetitive forms. So 




214 



T. Y. Nishida 



the search for language family which generates slender languages that are 
suitable to cryptography did not succeed. 

In this paper we introduce new controlled DTOL systems, called word 
length controlled DTOL systems, or wlcDTOL systems for short. A wlcDTOL 
system is a DTOL system added a control function which maps from the set of 
nonnegative integers to the set of tables. A wlcDTOL system selects the table 
or morphism to be iterated current word by the value of the control function of 
the length of current word. We show that every wlcPDTOL language is slender 
(Theorem 1). We give an example of wlcPDTOL system which generates a 
very complex language (Example 1). So the family of wlcPDTOL languages 
may be useful for cryptography. We consider other properties of wlcDTOL 
systems, including relationship between the family of wlcPDTOL languages 
and the family of DTOL languages and the finiteness problem for wlcDTOL 
languages. 

2 Preliminaries 

Let U he 8i finite alphabet. The element of U is called a letter. The set of 
all finite words over U including the empty word A is denoted by For a 
word w G U*, the length of w is denoted by lii;|. Let a be a letter in S. We 
denote by \w\a the number of occurrences of a in w. For a subset P C E, 
\w\p is the number of occurrences of letters in P, i.e., 

hip = E 

a€P 

Let 5 be an arbitrary set. The cardinality of S is denoted by card (5). 

We denote by E the set of nonnegative integers and E 4 . the set of positive 
integers. 

Let E and P be finite alphabets. A mapping h from E* to P* is said to 
be a morphism if h satisfies 



h{uv) = h{u)h{v) 

for every u,v G E*. A morphism from E* to E"^ is called a morphism over 
E. A morphism h is said to be A-free if for every a G E, h{a) ^ A. Let h be 
a morphism over E. For every n G H and w G E* , is defined by 

h^{w) = w and 

h'^{w) = h{h^~^(w)) for n > 0 . 

A triplet G = {E, /i, w) is said to be a DOL system if is a finite alphabet, 
/i is a morphism over 17, and w G E*. A DOL system G generates a sequence 
of words (wi) where W{ = h^{w). The language generated by G, denoted by 
L(G), is given by L{G) = {h^{w) > 0}. A DOL system is called a PDOL 

system if h is A-free. 
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A triplet G = {L,II,w) is said to be a DTOL system if is a finite 
alphabet, 77 is a finite set of morphisms over U, and w G U*. The language 
generated by G, denoted by L{G), is given by 

L{G) = {u\u = w or u = hi • - hk{w) where hi,. . . ,hk G 77}. 

A DTOL system is called a PDTOL system if every morphism in 77 is A-free. 

A language L over U is called slender if there exists a constant k G 
such that 

caxd{{w e L\\w\ = 1}) < k 

for every I G H. 

We assume the reader is familiar with the rudiments of the formal lan- 
guage theory and the theory of L systems, see, for example, [9,10]. 

3 Definition of word length controlled DTOL systems 

We give the central notion of this paper. 

Definition 1. A word length controlled DTOL system, or a wlcDTOL system 
for short, is a 4-tuple (T, 77, w, f) where Z* is a finite alphabet, 77 is a set of 
morphisms over D called the set of tables, t/; G TMs the axiom, and / is a 
partial recursive function from HI to 77 called the control function. 

A derivation by a wlcDTOL system is defined as follows. 

Definition 2. Let G = {D ,U,w, f) be a wlcDTOL system. Let x and y be 
words over D. Then G directly derives y from x if y = f{\x\){x). If f{\x\) is 
not defined, then G derives nothing from x. 

By Definition 2, a wlcDTOL system G = {E, 77, w, f) generates a sequence 
of words w = wo,wi, . . . ,Wi, . . . which is given by Wi.^i = f{\wi\)(wi) for 
i G U. The sequence (wi) is called the sequence generated by G. The language 
generated by G, denoted by L{G), is defined by 

L{G) = {t/ I is in the sequence generated by G}. 

A wlcDTOL system G = {E, 77, w, f) is said to be a wlcPDTOL system 
if every morphism h £ II is A-free. A language generated by a wlcDTOL 
(resp. wlcPDTOL) system is called a wlcDTOL (resp. wlcPDTOL) language. 
We denote by C{X) the family of X languages, where X is DTOL, PDTOL, 
wlcDTOL, or wlcPDTOL. 

Now we give an example of wlcPDTOL system. 

Example 1. Let G = {{A,a,b],{hi,h 2 },A,f) be a wlcPDTOL system where 

hi{A) — aA, hi(p) = a, hi{h) = b. 
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and 



/12(A) = bA, /12(a) = 6 , h‘2{b) = a 

f / \ _ / if a is a prime number 
^ ^ \ /i2 otherwise 

The first few words in the sequence generated by G is as follows: 



Wo = A, wi = /12(A) = 6 A, W‘2 = hi{wi) = baA, ws = hi{w2) = baaA, 

W4 = /i2(u^s) = obbbA^ — hi{w4) = abbbaA, wq — h2{w^) = baaabbA, 
W7 = hi (wq) = baaabbaA , .... 

Since / is a total recursive function, L{G) is infinite. We cannot characterize 
L(G) because we do not have an entire characterization of prime numbers. 

4 WlcDTOL languages and slender languages 

In this section we prove that every wlcPDTOL language is slender and that 
there is a wlcDTOL language which is not slender. 

First we establish a lemma concerning DOL sequence. 

Lemma 1. Let G = {E^h^w) be a DOL system with card(i 7 ) = k. Let (wi) 
be the sequence generated by G. If \wn\ = \wn-\-i\ = • • • = \wnJrk\ for some 
n > 0 , then \wn\ = \wi\ for every i >n. 

Proof Assume contrary, that is, if there exists an integer i > n such that 
l^il ^ l^nl = |w^n+i| = • • • = then by Theorem 1 . 3.6 of [ 9 ] (wj) is not 

a DOL sequence. □ 

Then we state the first main theorem of this section. 

Theorem 1. Every wlcPDTOL system generates a slender language. 

Proof. Let G = {E,U,w^f) be a wlcPDTOL system. We first note that the 
sequence w = wq,wi, . . . ,Wi, . . . generated by G satisfies |i«;o| < |^i| < • • • < 
\wi\ < • • •■ Since every finite language is slender, we assume L{G) is infinite. If 
L{G) is not slender, then for every k £ there exists i > 0 such that \wi\ = 
\wi^i I = • • • = \wi^k\- Because W{,. Wi^k have the same length, there is 
a morphism h £ II such that wj^i = h(wj) for j = i, . . . + k. In other 

words, the sequence is a PDOL sequence generated by 

(E,h,Wi). But by Lemma 1 , this is impossible. Thus L{G) is slender. □ 

The propagating restriction (that is, the assumption of wlcPDTOL sys- 
tem) of Theorem 1 is necessary. Indeed we have the next theorem. 

Theorem 2. There is a wlcDTOL language which is not slender. 
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In order to prove this theorem, we consider a wlcDTOL system Go = 
({A, B, a}, {hi, /i 2 }, A, f) where 

hi{A) = aA, hi{B) = aB, hi{a) = a, 



h2(A) = AB, h2{B) = B, h2{a) = X, 



and 



f{^) 






hi if X is not a perfect square number 
/i 2 if ar is a perfect square number 



Let Wo = A,wi, . . . ,Wi, . . . be the sequence generated by Gq. Then for every 
z > 0 the following properties hold: 



1. |w,U = 1- 



3. 









if \wi\ is a perfect square number 
otherwise 



if \wi\ is a perfect square number 
otherwise 



Now we establish three lemmas. 



Lemma 2. For every i > 0, there exists an integer m such that \wi\ = 
and 1 < m < 

Proof. We prove this lemma by induction on i. 

If i = 0, then the lemma holds with m = 1. 

Assume that the lemma holds for every nonnegative integer less than f 4- 1, 
i.e., \wi\ = m\wi\^yi^B} for some m. If \wi\ is a perfect square number, then 
|^i+i| = Hence the lemma holds with m = 1. If \wi\ is not a 

perfect square number, then \wi+i \ = \wi \ -f \wi\[A,B} = {'m-^l)\wi\^A,B}, in 
which the second equality is due to the inductive hypothesis. Moreover we 
have that and m < \wi\^A,B}’ Therefore \wi^i\ = 

(m + l)|i/;i+i and m -f- 1 < \wi^i □ 

Lemma 3. For every k > 0, Go generates a word w such that \w\[a,b} = k. 

Proof. We prove this lemma by induction on k. 

For fc = 1, the axiom A satisfies the result. 

Assume that the lemma holds for every positive integer less than A: -(- 1, 
i.e., \w\^a,b} — k for some w G L(Go). If \w\ is a perfect square number, 
then the word w' = h 2 (w) satisfies \w'\[a,b} = H{a,b] H- 1 = ik + 1. If |ti;| 
is not a perfect square number, then there exists a sequence of / -f 1 words 
w = • ,wi such that wi = hi(wi-i) and \wi\ is not perfect square 
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number for i = 1,2, . . . Then by Lemma 2, there is a positive integer m 
such that \w\ = m\w\^A,B}' By the property 2, we have 

IwiI = |w| + 

Because I is the largest number which is derived by hi from w and satisfies 
that \wi\ is not a perfect square number, we have that m4-/ + 1 = \'^\{a,b} 
and \wi^i\ = {m 1 + 1 )\w\[a,b} — (m -f i 4- 1)^. The word which is 
derived by hi firom wij^i^ i.e., wi^i = hi{wij^i)^ satisfies 

\'Wl+2\{A,B} = 1^/+1 \{A,B} H- 1 = \w\{a,B} -f 1 = 4- 1 

by property 2. Then the lemma holds for A: 4- 1. □ 

Let Pi , P ‘2 5 • • ■ 5 Pfc be arbitrary prime numbers such that pi < P 2 < • • • < 
Pk- Let P be the set of words which is given by P = {it; € L{Gq) | |it;| = 
P 1 P 2 * ' ’Pk}‘ We denote by \P\{a,b} fbe set of integers {liy|{A,B} \ '^ ^ ^}- 

Lemma 4. The set \P\{a,b} satisfies 



2 {P 1 P 2 • • ‘Pfc I (* 1 :* 2 , . . . ,ifc) e {1,2}* such that not all ij = 2}. 

Proof. By Lemma 3, for every (ii , 22 , . . . , ik) £ {1, 2}* such that not all ij = 2, 
there exists a word u E L{Go) which satisfies 

\u\{A,B} = Pi P^2 

We note that \u\[a,b} is not a perfect square number. Let ij = 2 — ij for 
j = 1,2,...,/?. Then there exists j such that ij = 1 because not all ij =2. We 
have a sequence of words u = uq,ui, . . . ,ui such that Ui^i = hi {u{), \ui \ is not 
a perfect square number for i = 0, 1, ...,/— 1, and m4-/ = P^iP^i * * •P^' where 
l^l = m\u\[A,B}- Therefore we have \ui\ = (m 4- 1)\u\^a,b} = PiPi ' "Pk-> i-^-j 
ui £ P. Since \u\{a,b} = \ui\{a,b} = • • • = K|{a,jb}, we have p^p"^ • • •p^*' G 
□ 

Proof of Theorem 2. For two words ii;i, ii ;2 E P, if |^21 {a,b} 5 then 

we have wi ^ wi. Now by Lemma 4, card(P) > card(|P|{yi ,B}) > 2* - 1. 
This means that L{Gq) is not slender. □ 

5 Other properties of wlcDTOL systems 

This section is devoted to the relationships among £(DT0L), £(PDT0L), and 
£(wlcPDT0L) and to the finiteness problem of wlcDTOL languages. First we 
consider a wlcPDTOL system which generates a language that is not DTOL 
language. 
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Example 2. Let G = {{o>}i{hi,h‘ 2 },a^, f) be a wlcPDTOL system where 
hi(a) — h‘ 2 {a) = and 

X _ ( hi if a: = 2^3^ for every n > 1 
^ ^ \ /i 2 otherwise 

Then G generates the wlcPDTOL sequence 

a^, h2{a^) = a^'^, hi{a^'^) = a^'^, 

Clearly L(G) = | n > 0} U I ^ > !}■ We denote L{G) by Li. 

We show that L\ is not a DTOL language. 

Lemma 5. The language L\ is not a DTOL language. 

Proof. If a DTOL system G = {{a},{hi,h 2 , . . . ,hk},w) generates Li, then 
w = and for every i = 1,2, . . . , /r, hi{a) ^ A because A ^ Li. Since a® is 
the second shortest word in Li, hj{a) = for some j. Then G derives 
for every i>l. This is a contradiction. □ 

Then we show that there is a PDTOL language which is not slender. 

Lemma 6. There is a PDTOL language which is not slender. 

Proof. Let G — {{a, 6, c}, {hi, / 12 }, cl) be a PDTOL system where 

hi{a) — ab, hi{b) = 6, hi{c) = c, 

h 2 (a) = ac, h 2 {b) = 6, / 12 (c) = c. 

Then it is obvious that L{G) = o{6,c}* is not slender. □ 

The above lemmas lead the next theorem. 

Theorem 3. We have the following relations: C(wlcPDTOL) and L (PDTOL) 
are incomparable and C (wlcPDTOL) and C(DTOL) are incomparable. 

Proof. By Lemma 5, £(DT0L) cannot include £(wlcPDT0L) and by Lemma 
6 and Theorem 1, £(wlcPDT0L) cannot include £(PDT0L). Then the results 
follow immediately. □ 

Now we discuss the finiteness problem for wlcDTOL languages. First we 
note that £ (wlcDTOL) is not include the family of all finite languages. 

Proposition 1. There is a finite language which is not a wlcDTOL language. 

Proof We will show that the language L 2 = {a^, a^} is not generated by any 
wlcDTOL system. If G = {{a}, U,w, f) generates L 2 , then for every h E II 
such that h is eventually used in the derivation, h is A-free because A ^ L 2 . 
Hence must be the axiom. But no morphism over {a} can obtain from 
a‘^. Thus L 2 is not a wlcDTOL language. □ 
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Then we show that the finiteness problem for wlcDTOL languages is un- 
decidable. 

Theorem 4. Finiteness problem for wlcPDTOL languages is undecidable. 

Proof. For every positive integer A:, we construct the wlcPDTOL system Gk = 
{{a,6},{/ii,/i2},a,/ib) where 

hi (a) = 6, hi (b) = 6, 

/12(a) = ab, h'zib) = 6, 

and 

f ( \ — j hi if X = k and TMk{k) is not defined 
otherwise 

in which T Mk is the one argument partial recursive function computed by 
the A:-th Turing machine under some numbering of Turing machines. Then 
fk is defined and has the value /i2 unless x = k and TMk{k) is not defined. 
Therefore L{Gk) is finite if and only if TMk{k) is not defined. This implies 
that an algorithm which solves the finiteness problem for wlcPDTOL lan- 
guages leads that the set {A: € \TMk{k) is not defined} is recursive. This 
is a contradiction (see, for example, [11]). □ 

Theorem 4 is rather obvious because the control functions may be partial 
recursive functions. If the control functions are restricted to total recursive 
functions, then it is interesting whether or not the finiteness problem for 
wlcDTOL languages is still undecidable. 
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Part V 



Algorithms and Complexity 




Program-Size Complexity of Initial Segments 
and Domination Reducibility* 



Cristian S. Calude and Richard J. Coles 



Summary. A theorem of Solovay states that there is a noncomputable A 2 real 
X = (xn)n such that H(xn) ^ H(n) 4* 0(1). We improve this result by showing 
there is a noncomputable c.e. real x with the same property. This answers a ques- 
tion concerning the relationship between the domination relation and program-size 
complexity of initial segments of reals (informally, a real x dominates a real y if 
from a good approximation of x from below one can compute a good approximation 
of y from below). Solovay proved that if x and y are two c.e. reals and y dominates 
X, then H{xn) ^ H{yn) -f 0(1). The result above shows that the converse is false, 
namely there are c.e. reals x and y such that H{xn) ^ H{yn) + 0(1) and y does 
not dominate x. 



1 Introduction 

Solovay [10] introduced the domination relation which plays an important role 
in defining the so-called i?-like reals. The class of 17-like reals coincides with 
the class of Chaitin 17 reals (halting probabilities of universal self-delimiting 
Turing machines [5-7, 2, 9]) (cf. [4]) and the class of c.e. random reals (cf. [11]). 
Solovay proved that if x and y are two c.e. reals and y dominates x, then 
H{xn) ^ H{yn) + 0(1). In this paper we prove that the converse implication 
is false, namely there are c.e. reals x and y such that H{xn) <F(y„)+0(l) 
and y does not dominate x. We do this by constructing a noncomputable 
c.e. real x such that H{xn) ^ H{n) -I- 0(1). In Section 2 we introduce our 
notation and basic concepts. Sections 3, 4 and 5 deal with constructing the 
c.e. real mentioned above. The methods of these sections generalise notions 
first appearing in Solovay [10]. In Section 6 we apply our result to the question 
of domination. 



2 Preliminaries 

Suppose a,6 E {0, 1}*, the set of binary strings. The concatenation of a, 6 is 
denoted by a^b. Let \a\ denote the length of a. For j ^ 0, we write a{j) = k 

* Calude was partially supported by AURC A18/XXXXX/62090/F3414056, 1996. 
Coles was supported by a UARC Post-Doctoral Fellowship. 
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iff the jth bit of a is fc. We let ^ denote the quasi-lexicographical ordering 
of finite binary strings. We write string (n) to denote the nth string with 
respect to ^ . By min^ we denote the minimum operation taken according 
to ^ . 

We fix a computable bijective function (•, •) from N x N to N. We write 
logn to denote log 2 n. For a function / : {0,1}* -> N, we define 0{f) = 
{g : {0,1}* N I 3c € NVa E {0,1}* {g (a) ^ c • /(a))}. Suppose ho, hi : 
{0, 1}* N. We write 0(1) for 0(f) when / is the constant function /(a) = 1 
for all a E {0, 1}*. We write ho ^ hi + 0(f) if there is a function g E 0(f) 
such that ho(a) ^ hi(a) -h g(a) for all a E {0, 1}*. 

We will look at real numbers in the interval [0, 1] through their binary 
expansions, i.e., in terms of functions n i-> Xn (from N into {0, 1}). We write 
(xn)n for the sequence n Xn- A real is computable if the function n i-)- is 
computable. A real x = (xn)n is computable enumerable (c.e.) if it is the limit 
of a computable, increasing, converging sequence of rationals. Equivalently, 
a = O.XA is a c.e. real if A has a computable approximation {A[s]} 5 ^o such 
that whenever i E A[s] and i ^ A[s 4- 1], then there is some j < i such that 
j ^ A[s] and j E A[s + 1]; is the characteristic function of A. Following 
Solovay [10] and Chaitin [5] we say that a real x dominates a real y if from a 
good approximation of x from below one can compute a good approximation 
of y from below (see Section 6 for a formal definition). 

Let (f)e be a standard list of all partial computable functions from N into 
N. In case 4>e(x) halts (and produces y) we write (f>e(x)l (<I>€(^)1= y)\ other- 
wise, <j)e(x)^. By (j)e(x)\t] we denote the time relativised version of (f>e(x), i.e., 
(f)e(x)[t] = 4>e(x) in case (f>e(x) halts in time t. The binary predicate </>g wwi 
is primitive recursive. We will adopt the following convention: if <?!>e (ar) [t] | , 
then (j)e(x) ^t} By dom</>e we denote the domain of the partial function (f>^. 

A self-delimiting computer is a partial computable function C from {0,1}* 
x{0, 1}* with values in {0,1}* such that for every y E {0,1}* the set {x | 
C(x, 2/)|} is prefix-free. Here C stands for the interpreter, x for the program, 
and y for the input data. 

The Invariance Theorem ([5,7,2,10]) states the existence of a universal 
computer U with the property that for every computer C there is a constant 
d (depending upon U and C) such that if C(x,y) = z, then U(x' ,y) = z, for 
some program x' with the length |x'| ^ |x| 4- d. Note that U does not need 
more than a primitive recursive extra time to simulate (7, i.e., there is a primi- 
tive recursive function h such that if C(x,y)\t]\^= z, then U (x' ,y)[h(t)]\^— z. 
In what follows we will fix a universal computer U. Let a* be the quasi- 
lexicographical least p such that U(p, A) = a. 



^ Sometimes we will be concerned with constructions occurring over o;-many stages 
and thus often append [t] to parameters to denote the value of the parameter at 
the end of stage t. 
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Define the following program-size complexities ([6]): 

H{a) = min{|p| | U(p,\) = a}, H{a,b) = H{{a,b)), 

H{a/b) = min{|p| | U{p,h'^) = a}, H{a/h) — min{|p| | U{p,b) = a}. 

Note that H{a/b^) = H{a/b). 

In expressions relating to strings, such as U{p,X) = n, we are identifying 
n with the binary string P of length n. We also write H(n) for i/(P). In 
fact notice that \H{n) — H{string{n))\ — 0(1). For integers n, we write n* 
for min^{p | U(p, A) = n}. 

We continue by defining some useful functions. For all j,n,t G N, define 
H(n)[t] = rain{|p| | U{p,X)[t] = n & |p| ^ t}, 
if such a p exists, H{n)[t] = 0, otherwise. Further let, 

a(n)[t] = min{if(j)[*] | j ^ n},a(n) = | j > n}. 

It is seen that H{n)[t] and a(n)[t] are primitive recursive functions (rea- 
son: for the computation we only need the set {p | t/(p, A)[t], \p\ ^ ^}), de- 
creasing in t, and H{n) = lim^ H{n)[t]^ a(n) = liiut a{n)[t]. 

Assume that D is an oracle and consider the relativised computation . 
Then the relativised program-size complexities are defined in the obvious 
way, for example. 



i/^(a) = min{|p||t/^(p, A) a}. 

If instead of a self-delimiting universal computer we work with a universal 
partial computable function V, then the induced complexities will be denoted 
by K{a),K{a/b)^K{a/b). We now summarise the known results relating the 
complexity of initial segments to the computability of a real. Let x = {xn)n 
be a binary sequence. 

Theorem 1 (Loveland [8]) x is computable iff K{xn/n) = 0(1). 
Corollary 2 x is computable iff H(xn/n) = 0(1). 

Theorem 3 (Chaitin [6]) x is computable iff K{xn) ^K{n) -{- 0(1). 
Theorem 4 (Chaitin [6]) If H{xn) ^ H{n) -h 0(1), then x G 
Proof. Start by noting that 

H{xn) = H{n,Xn) + 0(1) = H(xn/n) + H{n) -f 0(1). 

Now if X satisfies the hypothesis of the theorem, then we have 
H(xn/n) -b H(n) 4- 0(1) ^ H{n) -f 0(1). 
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Therefore H{xn/n) = 0(1), and so H^Xn/fi*) — 0(1). 

Relativising to the oracle D = {p\ U{p,X)l}, we have H^{n* /n) = 0(1), 
since the mapping n ^ n* is Turing reducible to D. Consequently, 

H^ixn/n) < H^{x„/n*)+H^{nyn) = 0 ( 1 ), 

therefore H^(x„/n) ^ 0(1). So by the relativised version of Corollary 2 
see that x is ^ 2 - 

Theorem 5 (Solovay [10]) There is a noncomputable real x such that 

H(xn) ^ H(n) + 0 ( 1 ). 

In section 5 we show the following stronger result: 

Theorem 6 There is a noncomputable c.e. real x such that for all n GN, 

H{xn) ^ H{n) + 0 ( 1 ). 

In Section 6 we make an application of this result to the domination 
degrees of c.e. reals. First we consider how to obtain a real with the desired 
initial segment complexity. 



we 

□ 



(W 



3 Achieving H(xn) ^ if(n) + 0(1) 

Suppose (ti)i is a computable increasing sequence of natural numbers. Define 
the total computable function a as follows: 

cr{i) - max{j ^ i \ H{j)[ti] = H{j)[ti+i]}. 

Notice that the graph of cr(i) is primitive recursive, and that cr(i) ^ i. 

Let (ti)i be a computable sequence of times and {pi | i ^ 0} a c.e. set of 
programs such that i <U < ti+i and U{pi, A)[fj] — i. For f € N define 

Ai = {p|f/(p,A)[tiU&|[/(p,A)Kf}. 

Notice that for every i £ N, Ai is computable, Ai C Ai+i, and for every 
n G N, n* e Aj+i \ Ai for some i. 

Proposition 7 If n* G Aj+i \ Ai, then a) n* G Aj for all j ^ i + 1, b) 
H{n)[t] = |n*| for all t ^ U+i, c) tr(i') > n for all i' '^i + 1. 

Definition 8 Let / : N N 6e o total computable increasing function. We 
say that a partial computable functionrli : {0,1}* —> {0,1}* is quasi-universal 
with respect to f */dom^ = doml7, and for all m,n G N, m <n implies that 
V’((m + /(m))*) tpiin + /(n))*). 
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Lemma 9 If is quasi-universal with respect to f then the real x = (xn)n 
defined by Xn = i’dn 4- /(n))*) satisfies the relation H{xn) ^H{n) 0(1). 

Proof. We have H{xn) ^ H{xn!n + /(n)) 4- H{n 4- /(n)) 4- 0(1). However 
H{n 4- f{n)) ^ H{n) -f 0(1) since / is computable, and 

H{xn!n + /(n)) = H{xn!{n + /(n))*) = 0(1), 

since Xn = '^((^ H- /(^))*) and ^ is partial computable with {n + f(n)Y in 
dom Therefore H{xn) ^ H{n) + 0(1) as required. □ 

Let / : N -> N be a total computable increasing function. Now suppose 
(z[i])i is a computable sequence of finite binary strings such that \z[i]\ ^ i 
and 

(n + /(n))* e ^,+1 \ Ai W > i, Vj ^ n {z[i + l](j) = z[i']{j)) (f) 

For p e {0,1}* define V’(p) as follows. If p e Ai+i \ Ai and |[/(p,A)| = 
n+f{n) for some n then let xl>(p) be the initial segment of z[i+\] of length 
n. Otherwise let be the initial segment oi z[i + 1] of length |17(p, A)|. 
Note that tj} is partial computable. 

Lemma 10 ip is quasi-universal with respect to f. 

Proof. Clearly ip has the same domain as U. So suppose m,n gN with m <n. 
Then ip{{n + f{n))*) is the initial segment of z[i + 1] of length n, where i + 1 
is sudi that (n + /(n))* € Ai+i \ Ai. But by (f), z[i + l](j) = z[i']{j) for all 
i' > i and all j ^ n. The same is true for m in place of n. Hence we must 
have ip{{m + f{m))*) 4 ip({n + /(n))*). □ 

So if we can construct a c.e. real z satisfying (f) for some suitable function 
/) then X = (xn)n = z defined by x„ = ip((n + /(n))*) will prove Theorem 6. 



4 Two Useful Results 

Before proving Theorem 6 we need two useful theorems, namely Theorems 
11 and 12 below due to Solovay [10]. Let B be a total increasing computable 
function which is not primitive recursive, but has a primitive recursive graph. 
For example take B to be Ackermann’s diagonal function. Then in fact, B 
dominates any primitive recursive function, that is, for every primitive recur- 
sive function g, B{n) > g{n), for almost all natural n; see [1]. 

Theorem 11 There is a computable sequence of times {tn)n, such that tn < 
^n+i for all n G N, and for all n > 0, 

(1) there is a natural number s„ such that B(t„_i) ^ s„ < B(s„) = t„, 
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(2) for all programs p with \p\ ^ U{p,X)l implies that either 

orU{p,X)[B{sn)]], 

(3) the predicate tn = j is primitive recursive, but the function n ^ tn is not 
primitive recursive. 

Proof. Let to = 0 and assume that si, . . . , Sn-i,ti, . . . ,tn-i have been de- 
fined. Let Q be the set of all programs of length less than or equal to 
B(tn-i). Let ri = tn-i and define the following three sets: Xi = {p G 
Q I t/{p,A)[n]i},yi = Q\Xi,Zi = {pe Fi I [/(p,A)[B(n)]i}. 

If Zi = 0, then let Sn = r\ and tn = B{sn)- Otherwise Z\ ^ 0, in which 
case set r *2 = B{ri) and define X 2 = U Zi,l2 ~ Q \ ^ 2 , ^2 = {p C I 2 | 
[/(p,A)[B(r2)U}. 

If Z ‘2 = 0, then let Sn = V 2 and tn = B{sn)- Otherwise continue this 
process until reaching a step i with Zi = ^ and hence let Sn = vi and tn = 
B{sn)- Such an i must be reached since X\ C X 2 C . . . C Q D Yi D Y 2 D . . . , 
Q is finite and Yi D Zi. 

It follows that (1) and (2) are satisfied. Part (3) follows from properties 
of Ackermann’s diagonal function B. □ 

We use the sequence of times {tn)n constructed above in the following 
theorem, which plays a crucial role in the priority argument in Section 5. 
In particular, the function a we work with is defined with respect to this 
sequence of times. 

Theorem 12 There is a computable sequence of times {ti)i^o, ti < tj+i 
and a c.e. set {pi}i^o of programs such that U{pi,X)[ti] = i, for which the 
following condition holds true: if g : N N is a total computable function, 
then for infinitely many natural numbers i, g{cr{i)) < i. 

Proof. Take the sequence of times (ti)i constructed in Theorem 11. Suppose 
for a contradiction that g{cr{n)) ^ n for almost all n. We may assume that g 
is increasing, and in fact that it dominates all primitive recursive functions. 

Let G{n) = min{m | g{m 4-1) ^ n}. Then our assumption on g im- 
plies that G{n) ^ (j(n) for almost all n. Note that G(n) ^ tn under this 
assumption.. 

The intuition for obtaining the contradiction is as follows. The function 
a{i), roughly speaking, is the largest j such that the approximation to H{j) 
does not change between time steps ti and ti+i. Therefore to show G{n) > 
a in) we would like to construct short programs (of length ^ B{tn)) for 
j G [G{n),n] which halt by time tn+i- We will be able to do this if 

(i) n has a short program that converges in approximately time tn, 

(ii) is small, 

because then we can use the Kraft-Chaitin Theorem to construct new codes 
(programs) for all j £ [G(n),n]. The next two lemmata perform this task. 
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Lemma 13 There is a primitive recursive function h such that for almost 
all k, there is a number Uk such that cx(G{nk))[tnk] ^ a program pn^ 

such thatU{pn^,\)[h{tni.)]l= nk, and |p„J = 0(logA;). 

Proof Fix A; G N. If n is sufficiently large then a(G(n)) ^ k because g is 
unbounded, and so a{G{n))[tn] ^ k. Let 

Uk = min{m \ m> k and a{G(rn))\tm] ^ k}. 



For large n, 



G{n) = min{m [ m ^ cr(n) and g(m + 1) ^ n}, 

so G is primitive recursive. Further, the predicate, o:(n)[t] ^ A: is also primitive 
recursive as it can be expressed by the formula 

(Vn ^j^t, \p\ ^ t{U (p, X)[t] =j=> \p\ ^ k)) or (n > t). 

So the function k Uk has a primitive recursive graph: U{p,\)[tm] = j 
iff [/(p, A)[s] = j, for some s ^ tm- Let hi{k,s) be the primitive recursive 
function evaluating the running time of the predicate rik = s. 

Now consider the self-delimiting computer ^0, A) = rik. Use the 

Invariance Theorem to show that for every natural k there is a program, p^^^ 
say, such that \pn^\ = 0{logk) such that Uipn^A) = rik- The time necessary 
to compute p^, from k is the sum between the time to get rik from k (a 
primitive recursive function in Uk, hi{k,rik)) and the time to simulate C on 
U, which is primitive recursive in the running time of C, i.e., h{hi(k,rik)). □ 

Lemma 14 For sufficiently large k, 

G(nk)^j^nk 

Proof Again, let k be sufficiently large so that a{G{rik)) ^ k. Suppose for a 
contradiction that 

G{nk)^j^rik 

The number Y^G{nk)^j^nk can be computed in a primitive re- 

cursive way and is less than 1. Computing its most significant k/2 digits 
we get a j E [G{rik),nk] and a program pj such that \pj\ ^ k/2 -h 0(1), 
^ (Pj^ U (pj, X)[z]l for some Consequently, H{j) ^ k/2 -h 0(1), 

so 

a{G(nk)) = min{i7(j) | j ^ G(nk)} ^ k/2 -f- 0(1). 

For sufficiently large A;, this contradicts a{G{rik)) ^ k, hence the required 
inequality has been demonstrated. □ 




232 



C. S. Calude and R. J. Coles 



We now continue with the proof of Theorem 12. Consider the following 
self-delimiting computer: 

(1) input string{k)^y, 

(2) compute Uk using the procedure in Lemma 13, 

(3) compute tn* and G{rik), 

(4) now use the Kraft-Chaitin Theorem^ to construct a prefix-free set E = 

[ej I j C [G{nk),rik]} such that \ej\ ^ - k/2, which is possible 

by Lemma 14, 

(5) output the code Cy of y if y E [G{rik),nk]- 

The time of this procedure is ho{tn^) for some primitive recursive function 
ho, and so ^ Hence for j C [G{nk),rik], 

^ H{j)[tn,] - k/2^0(l0gk). 

The O(logJk) term is derived from the length of the program in Lemma 13 
for computing Uk from k. Hence for sufficiently large fc, cr{nk) ^ G{rik), 
contradicting our assumption that G{n) < a(n) for almost all n. □ 

5 The Priority Argument 

We now return to the proof of Theorem 6. In this section we construct a 
noncomputable c.e. real that is consistent with the ideas of Section 3.^ 

Proof. We want to construct a c.e. real 2 ; such that the following requirements 
axe satisfied for all e > 0 : 

Re : 3m{<j)e(rn) ^ z{m)). 

We also want to ensure (f) is met so that the initial segments of z have the 
desired complexity. To understand how this is achieved, begin by thinking of 
the function / in (f) as the constant function 0. The idea is that once n* is 
in Aj+i then ( 7(5 H- 1) ^ n for all 5 > i. By using the function cr to provide 
witnesses for diagonalisation, we can ensure that the initial segment of the 
approximation to ^ of length n does not change after stage i because all future 
witnesses are greater than or equal to n. This method is essentially the one 
used by Solovay [10] to obtain a A 2 real x. We modify this to obtain a c.e. 
real 2 ; as follows. 

For a single requirement Ri in isolation, the naive approach is to wait 
for a stage s + 1 such that 4>i{a{s -f 1))| in less than s -I- 1 steps. Theorem 

^ Given a recursive list of “requirements” (n^, s*) {i > 0, ^ 0) such that 
Yli ^ 1? we can effectively construct a self- delimiting computer C and a 
recursive one-to-one enumeration xo, xi, X 2 , . . . of words xi of length m such that 
C(xi, A) = Si for all i and C(x, A) = 00 if x ^ {xi | i ^ 0}; see [2]. 

^ A full proof of Solovay’s Theorem 5 can be found in Calude and Coles [3] and 
may help the reader to understand the c.e. case below. 
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12 tells us that there are infinitely many such stages if <^e is total. Now set 
z(cr(s + 1)) + 1)). 

For two requirements, Ri of higher priority than R 2 , we may diagonalise 
against ^2 on argument a{s -h 1) only to find at a stage 5 ' -h 1 > 5 + 1 
that ^( 5 ' -I- 1) = cr(5 4- 1) and we would like to diagonalise against 4>i on 
argument cr(s' -f 1). If z[s 4- l](<r(5 4- 1)) was set to 1 and we want to set 
z[s‘ -f 1 ](<t( 5' 4- 1)) to 0 then we risk not constructing z to be a c.e. real. To set 
z[s‘ 4- l](t^(5 -h 1)) = 0 we must find some c < a{s' 4* 1) such that z[s']{c) = 0. 
Then we can define z[s' 4- l](c) = 1 allowing us to set z[s' 4- l](cr(5' -h 1)) = 0 
so that 2 ; is c.e. Thus before using a{s -h 1) as a witness for R 2 we must make 
sure there is some c < cr(s 4- 1) which is available for correction if it turns 
out Ri would like to use (j{s 4- 1) as a witness at some later stage. Therefore 
in the construction, R 2 may only diagonalise with ( 7(5 4- 1) if cr (5 4- 1) >2. 

For an arbitrary requirement Rg, this generalises to Rg using witnesses 
(j( 54-1) only if <j(5-l-l) > e-\-w{e)[s-\-l]^ where w(e)[s+l] is the largest witness 
seen so far for requirements Rg/ such that e' ^ e. This ensures enough space 
is left for correction in case higher priority requirements require witnesses 
used by lower priority requirements. 

If a higher priority requirement wants to use a witness or(s 4- 1) that has 
already been used by a lower priority requirement, but z[5](o-(5 -h 1)) = 0, 
then of course the higher priority requirement could set ^[5 4- 1]((t( 5 4- 1)) = 1 
without using a correcting bit c and still keep 2 ; c.e., but to simplify the 
presentation we always use a correcting bit c whenever a requirement receives 
attention. Consequently, we insist witnesses used for Rg are actually greater 
than 2e-\-w{e)[s-\-l] so enough space is available for correction. This approach 
means we can reset z[s]{j) = 0 for all j > c such that j o-(5 + 1) whenever 
we act for some requirement at stage 5 4-1. 

The need to leave space for correcting bits to make z a c.e. real is the 
reason for the computable function / mentioned in (f). The particular / we 
use is described after the construction of z. The idea is that we have to look 
beyond n to find out when an initial segment of z of length n has settled 
down. That is, once (n 4- f(n))* C \ Ai, then cj(i') ^ n 4- /(n) for all 
i' > i. Therefore if /(n) is large enough, all changes of bits of z will be above 



The Construction. 

Stage s = 0: Let z[0] = A. 

Stage s 4- 1: We have already defined z[s]. 

We say that Rg requires attention at stage 5 4- 1 if 

(i) e < 5 4- 1, 

(ii) (j>e{cr{s 4- 1))J, in at most 5 + 1 steps, 

(iii) a{s 4- 1) is not a witness for any requirement Rg/ with e' < e, 

(iv) Rg has no witness at stage s -f 1 and ^ 7(5 + 1) was not previously a 
witness of Rg , 
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(v) cr{s 4- 1) ^ 2e -i-w{e)[s 4- 1], where w{e)[s 4- 1] is the largest witness used 
so far for requirements Re' with e' ^ e, if it exists, and w{e)[s 4- 1] = 0 
otherwise. 

We act for the least e such that Rg requires attention, eo say. We say that 
Reo receives attention at stage s + 1 and is satisfied. 

Let c be the largest natural number such that w{e)[s 4- 1] < c < a{s 4- 1), 
z[s]{c) = 0 and c is not a witness for a requirement Rg' for all e' < e. (The 
existence of such a c is proven below in Lemma 15.) Define 

z[s + l](c) = 1, 

z[s + 1](ct(s + 1)) ^ (peoif^is + l))i, 
z[s 4- l](i) = 0 for all j > c such that j ^ a{s 4- 1), 
z[s 4- l](j) = z[s](j) for all j < c. 

Finally cancel all witnesses for requirements Rg/ for all e' > eo- Declare 
Rg' unsatisfied for all e' > eo- This completes the construction. 

The Verification. 

Lemma 15 If Re receives attention at stage 5 4-1 then there is some c with 
ty(e)[5 4- 1] < c < a{s 4- 1) such that ^[5](c) = 0 and c is not a witness for 
requirements Rg/ with e' < e. 

Proof. Suppose not for a contradiction and let 54-1 be a least counterexample. 
Suppose w(e)[s 4- 1] is chosen as a witness for Rg/ where e' ^ e at a stage 
5 ' 4- 1 < 5 4-1. Let 5o 4- 1 be the largest stage < 5 4- 1 for which some 
requirement Rg/ with e' < e received attention. Then there is a c < a{so 4- 1) 
such that -2;[5o](c) = 0 and c is not the witness for a requirement Rg// for all 
e" < e' at stage sq 4- 1. Then z[so -h l](j) = 0 for all j > w{e)[so 4- 1]. 

We now consider what can happen between stages 5 q 4- 1 and 5 4-1. As 
no requirement of higher priority than Rg receives attention between stages 
5o 4- 1 and 5 4-1, the only requirements that can change the value of bits 
in the approximation to 2 ; of less than cr{s 4- 1) are those of lower priority 
than Rg. But by construction a lower priority requirement R^, i > e, leaves 
at least 2i bits between any witness it may use and w{e)[s + 1]. Hence, there 
is a c such that w{e)[s 4- 1] < c < cr{s 4- 1) and -2;[5](c) = 0 and c is not the 
witness for a requirement Rg/ , e' < e. (Note that < 7(5 4- 1) is strictly greater 
than w{e)[s 4- 1] as by construction, cr{s 4- 1) ^ w{e)[s 4- 1] 4- 2e.) □ 

Lemma 16 For all e G N, Rg has at most a finite number of witnesses. 

Proof An easy induction on e shows that Rg can only require attention 
finitely often. □ 



Lemma 17 limsz[5](z) exists for all i and z ts a c.e. real. 
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Proof. This follows from Lemmas 15 and 16, and by construction. □ 

Lemma 18 If (f>e total then Rg has a final witness. 

Proof. Suppose Rg does not get a final witness. Let sq be the least stage > e 
such that for all e' < e, Rg/ does not require attention at all stages s sq. 
Then clearly (i) holds for all stages s'^ sq. Choose si > sq so that (iii) and 
(v) hold for all stages s si. This is possible since lim^ a{s) = oo. Now let 
g{m) be the number of steps it takes for </>g(m) to converge. Then (ii) holds at 
infinitely many stages since there are infinitely many i such that g{cr{i)) < i 
by Theorem 12. Hence (iv) must fail, for otherwise Rg would receive attention 
at some stage «2 ^ ^ind get a final witness. Therefore Rg has a witness at 
a stage after si which causes (iv) to fail, but then this witness is permanent, 
contradicting the assumption. Hence Rg has a final witness. □ 

Lemma 19 If m is the final witness of Re, then <l)e(rn) z{m). 

Proof. By construction, if m is the final witness of Rg then z{m) ^ (j>e{m). □ 

It remains to show that the initial segments of z of length n have com- 
plexity ^ H{n) + 0(1). 

We inductively define the computable function / : N N as follows: Run 
the construction above looking for a stage s 4- 1 such that: 

(1) a(s -f 1) is used as a witness for some requirement Rg, 

(2) a{s -h 1) - n > /(m) for all m < n, 

( 3 ) a(s -h 1) > n -h 2e, 

(4) Upon finding such a (t(s 4- 1), set f(n) = a(s -h 1) - n. 

We remark that ( 3 ) is possible by waiting for a stage s 4- 1 where w(e)[s + 
1] > n. Then / is a total computable and increasing function and {z[i])i is a 
computable sequence. Define : {0,1}* {0,1}* as in Section 3, namely: 

for p E {0, 1}*, wait for p E >1*4-1 \ >1*. Then t/(p, A)| and \U{p,X)\ ^ i by 
definition of >li-fi. If \U{p,X)\ = f{n) + n for some n ^ i, then let F(p) be 
the initial segment of z[i 4- 1] of length n. Otherwise let F(p) be the initial 
segment of z[i 4- 1] of length \U(p,X)\. 

If we can show that / and {z[i])i meet (f) then by Lemma 10 is quasi- 
universal with respect to /. Furthermore, Lemma 9 proves that defining x by 
Xn = 'fpi'n f{n)y) satisfies Theorem 6. 

Lemma 20 Suppose that (nH-/(n))* is in Ai^i \Ai. Then for all j ^ n and 
V > i, z[i + l](j) = z[i']{j). 

Proof. First notice that since n 4- /(n) was a witness for Rg, then all lower 
priority requirements (Rg/ for e' > e) use witnesses greater than 2e' + (j(s-f 1) 
by construction. Hence if action is taken for a requirement Rg/ with e' > e at 
astages'4-1 > s4-l then there is always a c such that cr(s 4-1) < c < cr(s'4-l) 
that can be used for correction by Rg/ . 
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Thus we only need to consider requirements Re' for e' ^ e. In order that 
z[i + l]{j) i=- z{j) for some j ^ n, some witness y f{n) -\-n must be used 
by a requirement Re' for some e' < e at a later stage, s\ > i say. 

Now since (n 4- /(n))* € \ At, (j(s') ^ n + /(n) for all s' ^ i 4- 1. At 

stage s 4-1 cr(s4-l) was used as a witness and hence cr{s + l) > 2e-l-zi;(e)[s + l]. 

Let So + 1 be the largest stage < s 4- 1 such that some requirement Re' 
with e' ^ e received attention. Then z[sq 4- l](j) = 0 for all j > w{e)[s 4- 1] 
by construction and Lemma 15. 

There are e — 1 requirements of higher priority than e, and each may 
want to use (t(s -h 1) as a witness at some stage after stage s 4- 1. However 
cr(s H- 1) > n H- 2e, and so if such action is taken at a stage s' 4- 1 > s + 1 for 
a requirement Re' where e' < e, then there is a c such that n < c < a{s 4- 1), 
z[s']{c) = 0 and c is a not a witness for a requirement Re" for e" < e'. 

So (t) is satisfied. Hence n < m implies that F{n*) =4 F{m*). and Xn = 
F(n*). □ 

This completes the proof of Theorem 6. □ 

6 Domination Degrees and Program-size Complexity 

A real x is said to dominate the real y if there is a partial computable function 
/ from rationals to rationals and a constant c > 0 with the property that 
if p is a rational number less than x, then f{p) is (defined and) less than 
y, and it satisfies the inequality c{x — p) ^ y — /(p)- In this case we write 
X '^dom y- A c.e. real is universal if it dominates every c.e. real; such a real 
is called Solovay-Chaitin i?-like [10,5]. The class of Chaitin Q reals [5,7,9] 
coincides with the class of Solovay-Chaitin i?-like reals, see Calude, Hertling, 
Khoussainov, Wang [4], which equals the class of c.e. random reals, Slaman 
[11]. For the theory of randomness see Chaitin [7] and Calude [2]. 

Does domination reducibility capture randomness of c.e. reals? On the 
one hand we have: 

Theorem 21 (Solovay [10]) Let x and y be two c.e. reals. Then x ^dom y 
implies H{xn) ^H{y„) + 0{1). 

So, the natural question is whether the converse implication is true. We 
can now show that the answer is negative. 

Theorem 22 There exist c.e. reals x and y such that y '^dom x and H{xn) ^ 
F(yn)4-0(1). 

Proof. Following Theorem 6 let x be a noncomputable c.e. real such that 
H{xn) ^ H{n) 4- 0(1). Let y be any computable real. Then x ^dom y but 

y '^dom X. 

Now since y is computable, H{yn) = H{n) 0(1). Furthermore, since 
X ^dom y we have H(yn) ^ H{xn) + 0(1) by Theorem 21. Suppose that 
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there is no constant c such that H{xn) Then for every constant 

c there are infinitely many n such that H{xn) > H{yn)+c. Consequently for 
every constant c there are infinitely many n such that H{xn) > H(n) 4- c, a 
contradiction. Therefore H{xn) ^ H(yn) + 0(1). □ 
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stability of Approximation Algorithms 
and the Knapsack Problem 



Juraj Hromkovic 



Summary. The investigation of the possibility or the impossibility to efficiently 
compute approximations of hard optimization problems becomes one of the cen- 
tral and most fruitful areas of current algorithm and complexity theory. Currently, 
optimization problems are considered to be tractable if there exist randomized 
polynomial-time approximation algorithms that solve them with a reasonable ap- 
proximation ratio. Our opinion is that this definition of tractable problems is still 
too hard. This is because one usually considers the worst-case complexity and what 
is really important is the complexity of algorithms on “natural” problem instances 
(real data coming from the practice). Nobody exactly knows what “natural” data 
are and how to mathematically specify them. But, what one can do is to try to 
separate the easy problem instances from the hard ones. The aim of this paper is 
to develop an approach going in this direction. 

More precisely, a concept for measuring the stability of approximation algo- 
rithms is presented. This concept is of theoretical and practical interest because it 
can be helpful to determine the border between easy problem instances and hard 
problem instances of complex optimization problems that do not admit polynomial- 
time approximation algorithms. We illustrate the usefulness of our approach in an 
exemplary study of the knapsack problem. 



1 Introduction 

Immediately after introducing NP-hardness (completeness) [5] as a concept 
for proving intractability of computing problems [13], the following question 
has been posed: If an optimization problem does not admit an efficiently com- 
putable optimal solution, is there a possibility to efficiently compute at least 
an approximation of the optimal solution? Several researchers [4], [10], [11], 
[12], [14], [19], [20] provided already in the middle of the seventies a positive 
answer for some optimization problems. It may seem to be a fascinating effect 
if one jumps from the exponential complexity (a huge inevitable amount of 
physical work) to the polynomial complexity (tractable amount of physical 
work) due to a small change in the requirement - instead of an exact optimal 
solution one forces a solution whose quality differs from the quality of an 
optimal solution at most by e • 100 % for some e. This effect is very strong, 
especially, if one considers problems for which this approximation concept 
works for any small e (see the concept of approximation schemes in [10], [16], 
[18], [3]). 
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There is also another possibility to jump from NP to P. Namely, to con- 
sider the subset of inputs with a special, nice property instead of the whole 
set of inputs for which the problem is well-defined [13]. A nice example is 
the Travelling Salesman Problem (TSP). TSP is not only NP-complete, but 
also the search of an approximation solution for TSP is NP-complete for ev- 
ery e} But if one considers TSP for inputs satisfying the triangle inequality 
(so called Zi-TSP), one can even design a polynomial-time approximation 
algorithm [4] with the quality e = ^. The situation is still more interesting, 
if one considers the Euclidean TSP, where the distances between the nodes 
correspond to the distances in the Euclidean metrics. The Euclidean TSP is 
NP-complete [17], but for every small e > 0 one can design a polynomial- 
time e-approximation algorithm [1], [2], [15], whose randomized version has 
an almost linear time complexity.^ 

The fascinating observations of huge quantitive changes mentioned above 
led to the development of the concept of ’’stability” of approximation algo- 
rithms in [9]. In this paper we give the formal description of this concept. 
Then, we apply it for the knapsack problem in order to illustrate its useful- 
ness. 

Informally, one can introduce the measurement of the stability of approx- 
imation algorithms as follows: Let us consider the following scenario. One has 
an optimization problem P for two sets of inputs Li and I/ 2 , Li C L 2 . For 
Li there exists an polynomial-time ^-approximation algorithm A, but for L 2 
there is no polynomial-time (^-approximation algorithm for any S > 0 (if NP 
is not equal to P). We pose the following question: Is the algorithm A really 
useful for inputs from Li only? Let us consider a distance measure M in L 2 
determining the distance between Li and any given input x £ L 2 - L\. Now, 
one can consider an input x £ L 2 -L 1 , for which there distanceM{x, Li) <k 
for some positive real k. One can look for how ’’good” the algorithm A is for 
the input x £ L 2 — Li. li for every A: > 0 and every x with the distance at 
most A; to Li, A computes an approximation of an optimal solution for 
X {Se^k is considered to be a constant depending on k and e only), then one 
can say that A is ’’(approximation) stable” according to the distance mea- 
sure M. Obviously, such a concept enables to show positive results extending 
the applicability of known approximation algorithms. On the other hand it 
can help to show the boundaries of the use of approximation algorithms and 
possibly even a new kind of hardness of optimization problems. 



^ In fact Hastad [7] even proved that there is no polynomial-time 
approximation algorithm for TSP for any 6 > 0 {n is the input size) if NP is 
not a subset of ZPP. 

^ Obviously, we know a lot if similar examples where with restricting the set of 
inputs one crosses the border between decidability and undecidability (Post Cor- 
respondence Problem) or the border between P and NP (SAT and 2-SAT, or 
vertex cover problem). 
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This paper is organized as follows: In Section 2 we present the concept 
of approximation stability. In order to do this, we give a new definition of 
optimization problems that enables to investigate some new aspects of opti- 
mization problems. In Section 3 we investigate the stability of the well-known 
PTAS [19] for the simple knapsack problem according to a reasonable dis- 
tance measure. We do not prove any new result for the knapsack problem 
because we have already efficient FPTAS for it [10]. But the aim is to present 
an example that exemplary shows the usefulness of our concept. In the Icist 
Section 4 we discuss the possibilities of the use of the presented concept in 
the investigation in the area of combinatorial optimization. 

2 Definition of the Stability of Approximation 
Algorithms 

We assume that the reader is familiar with the basic concepts and notions of 
algorithmics and complexity theory as presented in standard textbooks like 
[3], [6], [8], [18], [21]. Next, we present the new definition of the notion of an 
optimization problem [9]. The reason to do this is to obtain the possibility 
to study the influence of the input sets on the hardness of the problem con- 
sidered. Let IN = {0, 1,2, ...} be the set of nonnegative integers, and let 1R“^ 
be the set of positive reals. 

Definition 1. An optimization problem 1/ is an 7-tuple U = (17/ , Eq, 
L, L/, M, cost, goal)^ where 

1. 27/ is an alphabet called input alphabet, 

2. 27o is an alphabet called output alphabet, 

3. L C i7j is a language over 27/ called the language of consistent inputs, 

4. L/ C L is a language over 27/ called the language of actual inputs, 

5. A4 is a function from L to 2^o ^ where, for every x £ L, M{x) is called 
the set of feasible solutions for the input x, 

6. cost is a function, called cost function, from [J^^eL 

7. goal £ {minimum, maximum}. 

For every x £ L, we define 

Outputu{x) = [y £ M{x)\cost{y) = goal [cost {z)\z £ A4(x)}} , 



and 

for some y £ Outputu{x). 



Optu{x) = cost{y) 



Clearly, the meaning for 27/, 27o, M, cost and goal is the usual one. L 
may be considered as a set of consistent inputs, i.e., the inputs for which the 
optimization problem is consistently defined. Lj is the set of inputs consid- 
ered and only these inputs are taken into account when one determines the 
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complexity of the optimization problem U. This kind of definition is useful 
for considering the complexity of optimization problems parameterized ac- 
cording to their languages of actual inputs. In what follows Language{U) 
denotes the language Lj of actual inputs of U. 

To illustrate Definition 1 consider the Knapsack Problem (KP). In what 
follows bin{u) denotes the integer binary coded by the string u E {0, 1}*. 
The input of KP consists of 2n 4- 1 integers w\^ W 2 j Wn, b, ci, C2, 

Cn, n € IN. So, one can consider Ej = {0,1,#} with the binary coding 
of integers and # for The output is a vector x E {0,1}^, and so we 
set Eo = {0, 1}. L = (0, 1}* • U£o 1}*)'^*. We speak about the Simple 
Knapsack Problem (SKP) if Wi = Ci for every i = 1, ..., n. So, we can consider 
Li = (zi#2:2#...#2;n#6#2;i#2;2#...#^n G {0,1}* for i = l,...,n, n E 
IV} as a subset of L. M assigns to each I = yi#...#2/yi#6#iii#...#i/^ 
the set of words M(I) = {x = XiX 2 ...Xn e {0,1}"" I ‘ < 

bin(b)}. For every x = xi...Xn E M(I), cost(x) = * bin(ui). The 

goal is maximum. So, KP = {Ej, Eq,L,L, M, cost, goal), and SKP — 
(Ej, Eo, L, Li, M, cost, goal). 

Definition 2. Let U = {Ei,Eo,L,Li,M, cost, goal) be an optimization 
problem. We say that an algorithm A is a consistent algorithm for U 
if, for every input x £ Li, A computes an output A{x) E M(x). We say 
that A solves U if, for every x £ Li, A computes an output A{x) from 
Outputu{x). The time complexity of A is defined as the function 

TimcAin) = max{TimeA{x)\x £ Li C\ Ej} 
from IV to IV, where TimcAix) is the length of the computation of A on x. 

Next, we give the definitions of standard notions in the area of approxi- 
mation algorithms. 

Definitions. Let U = (Ei,Eo,L,Li,M, cost, goal) be an optimization 
problem, and let A be a consistent algorithm for U. For every x £ Li, the 
relative error €a(x) is defined as 

* /_X _ \cost{A{x)) - Optv{x)\ 

o^) • 

For any n e ]N, we define the relative error of A 

£A(n) = max{e^(x)|x G L/ D E^}. 

For any positive real 5, we say that A is an ^-approximation algorithm 
for U if e>i(x) < 8 for every x G L/. 

For every function / : IN ]R, we say that A is a /(n)-approximation 
algorithm for U if £yi(n) < /(n) for every n G IN. 
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The best what can happen for a hard optimization problem U is that one 
has a polynomial-time e-approximation algorithm for U for any e > 0. In 
that case we call this collection of approximation algorithms a polynomial- 
time approximation scheme (PTAS) for U. A nicer definition of a PTAS 
than the above one is the following one. An algorithm jB is a PTAS for an 
optimization problem U B computes an output B(x,£) G M for every 
input (x,e) G Language{U) x with 

|cost(J9(x,g)) - Opt{x)\ ^ 

Opt{x) ~~ ^ 

in time polynomial according to lx|. 

Now, we define the complexity classes of optimization problems in the 
usual way. 

Definition 4. 

NPO = {U = {Bi, Bo, L, Lj.MyCost, goal) 1 is an optimization 
problem, L,L/ G P; for every x £ L,M{x) e P\cost 
is computable in polynomial time} , 

For every optimization problem U = (Z*/, Bq, T, Lj, M, cost, goal), the 

underlying language of U is 

Underu = k)\w G Lj, k — {0}, Optu{w) < k} 
if goal — maximum. Analogously, if goal = minimum 

Underu = {{'^,r) | w G Lj, r G IN - {0}, Optu{w) > r} . 

PO = {U e NPO I Underu € P} 

APX = [U e NPO 1 there exists an ^-approximation algorithm 
for U for some e G IR^} . 

In order to define the notion of stability of approximation algorithms we 
need to consider something like a distance between a language L and a word 
outside L. 

Definition 5. Let U = {Bj, Bq, L, Lj, M, cost, goal) and U = {Bj, Bq, 
L, L, M, cost, goal) be two optimization problems with Lj CL. k distance 
function for U according to Lj is any function hi ■ L ^ satisfying 
the property 

hii^) = 0 for every x e Lj . 

We define, for any r G IR"^, 

Roundr,h{Li) = {w £ L \ h{w) <r}CL. 
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Let A be a consistent algorithm for V, and let A be an e-approximation 
algorithm for U for some £ E IR"^. Let p be a positive real. We say that 
A is p-stable according to h if, for every real 0 < r < p, there ex- 
ists a 6r,e € such that A is an -approximation algorithm for = 
(17/, I7o, L, Raandr^h{Li),M, cost, goal).^ 

A is stable according to h ii A is p-stable according to h for every 
p E We say that A is unstable according to hit A is not p-stable for 
any p E IR”^ . 

For every positive integer r, and every function —> IR*^ we say that 

A is (r, /(n))-quasistable according to h if A is an /y.(n)-approximation 
algorithm for Ur = (^/, L, Roundr^h{Li), M, cost, goal). 

Note, that considering PTASs the situation may be a little bit more 
complicated. Let us consider a PTAS A as a collection of plynomial-time 
^-approximation algorithms A^ for every e > 0. If is stable according to 
a distance measure M for every e > 0, then it does not immediately im- 
ply that A is a PTAS for inputs from Ur€iN happens 

if, for instance, 5r,e = r + e. Then Ag is a (r -h ^)-approximation algorithm 
for Roundr,M{Li), but we have obtained no PTAS for Roundr,M{Li). Obvi- 
ously, if = 6 • r, then A is a PTAS for Roundr,M(Li). This is the reason 
to define the stability of PTASs as follows. 

Definition 6. Let U,U, Ur and h have the same meaning as in Definition 
5. Let A as collection of algorithms {Ae}g>o be a PTAS for U. If, for every 
r > 0, Ag is a (Sy.,g-approximative algorithm for [7^? we say that A is stable 
according to h. 

If < /(^) • g{r), where 

1. / and g are some functions from IR to IR*^, and 

2. lime_^o f(e) = 0, 

then we say that the PTAS A is super-stable according to h. 

One may see that the notions of stability can be useful for answering the 
question how broadly a given approximation algorithm is applicable. So, if 
one is interested in positive results then one is looking for a suitable distance 
measure that enables to use the algorithm outside the originally considered 
set of inputs. In this way one can search for the border of the applicability of 
the given algorithm. If one is interested in negative results then one can try 
to show that for any reasonable distance measure the considered algorithm 
cannot be extended to work for a much larger set of inputs than the origi- 
nal one. In this way one can search for fine boundaries between polynomial 
approximability and polynomial non-approximability. A more involved dis- 
cussion about the applicability of our concept is presented in the last section 
of this paper. 

^ Note, that 6r,e is a constant depending on r and e only. 
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3 An Example — Stability of Approximation and the 
Knapsack Problem 

In this section we consider the stability of the well known PTAS [19] for the 
simple knapsack problem (SKP). This PTAS is a combination of (compromise 
between) the total search and the greedy method. We show that this PTAS is 
stable according to a reasonable distance measure. This leads to an approx- 
imation algorithm for some extension of the simple SKP, but unfortunately 
not to a PTAS for this extension. This is the motivation for modifying this 
PTAS in order to obtain a PTAS for the extension according to the distance 
measure considered. 

Note, that we do not develop any new algorithm here [19] and that one 
knows already a FPTAS for the general knapsack problem [10]. The aim of 
this section is only to illustrate how the study of the stability of an approx- 
imation algorithm (or a PTAS) may lead to the development of a modified 
version of this algorithm (PTAS) that can be applied for inputs for which the 
original algorithm cannot be successfuly applied. 

First, we review the PTAS for the simple knapsack problem. 



PTAS SKP 



Input: Positive integers wi,W2, ...,Wn,b for some n € IN and some positive 
real number 1 > £ > 0. 

Step 1: Sort w\,W 2 , For simplicity we may assume wi > W 2 > ... > 

Wn. 

Step 2: Set k = fl/e]. 

Step 3: For every set T = {* 1 ,^ 2 , Q {1,2, ...,n} with \T\ = I < k and 
^ extend T to T' by using the greedy method and values 
Wii^i,Wii^ 2 ,.:,Wn. (The greedy method stops if and wj > 

^ - YlieT' 3 iT', j> ii.) 

Output: The best set T' constructed in Step 3. 



For every given e, we denote the above algorithm by SKP^. It is known 
that SKPg is an ^-approximation algorithm for SKP. Observe that it is con- 
sistent for KP. Now, we consider the following distance function DIST for 
any input wi^ W 2 , Wn, b, ci, ..., Cn of KP: 

DIST(w\y ...yWfi^by Cfi) — 



max 




Ci >Wi, i £ {1, 
Wi >Ci,i£ ( 1 , 




Let KPs = L, Rounds^DiST{Lj),M, cost, maximum) for any 6. 

Now, we show that PTAS SKP is stable according to DIST but this result 
does not imply the existence of a PTAS for KPs for any (5 > 0. 
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Lemma 1. For every e > 0, J > 0, the algorithm SKP^ is an (e S(2 S) • 
(1 + e))- approximation algorithm for KP^. 

Proof. Let w\ > W 2 > ... > Wn for an input I = wi, ...,Wn,b,ci, and 

let k = Let U = {ii, 2 * 2 , ...,«/} C {1,2, ...,n} be an optimal solution 

for /. If Z < A; then SKP^ outputs an optimal solution with cost{U) because 
SKPg has considered Z7 as a candidate for the output in Step 3. 

Consider the case I > k. SKPe has considered the greedy extension of 
T = in Step 2. Let T' = ...,u, jAr+i, ..., jj^+r} be the 

greedy extension of T. Obviously, it is sufficient to show that the difference 
cost(U) — cost{T') is small relative to cost(U), because the cost of the output 
of SKPg is at least cost(T'). We distinguish the following two possibilities: 

1. Let '^3 ^ Obviously, for every i, + < 1 + 

S. So, costiU) = 2 a < (1 + (5) • Wi and cost{T) = > 

(1 + S)~^ • '^ 3 - ^Lis way we obtain 



cost{U) - cost{r) < (1 + (5) • - (1 + 5)~^ 



ieu 



<il + S)-'^Wi-{l + S)-^ 
ieu 

_ 8-{2 + 6 ) 



l + S 



Wi 



ieu 



ieu 

= 6-(2 + 6)-^Ci 

ieu 

= S- (2 + S)- cost(U) . 



H “'i 

jeT' 

ieu 



Finally, 



costjU) - cost{V) ^ (5 • (2 -h (J) • cost{U) 
cost{U) ■“ cost(U) 



— 5 • (2 H" (5). 



2. Let d — Ylieu'^i ~ ^^eT' ^ Let c be the cost of the first part 

of U with the weight Then in the same way as in 1. one can 

establish <6- (2 + 5). 

It remains to bound cost(U) - c, i.e. the cost of the last part of U with 
the weight d. Obviously, d<b- for some r>k,ir^U 

(if not, then SKP^ would add r to T' in the greedy procedure). Since 

Wi^ ^ ^ ^ Wij^ , 

d<wi^ < + + < T.ieu^i 



<e-Y^Wi. 



r 
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Since cost(U) < c + d • (1 + (5) we obtain 



cost{U) — cost{T') ^ c -h d • (1 4- (S) — cosi{T') 
cost{U) ~ cost{U) 

^ c - costjT') (l + (^) ■e-'Eieu'^i 
~ cost{U) cost{U) 

< (5 • (2 + J ) + (1 + ( S ) . £ • (1 + ( 5 ) 

= £ -h (5 • (2 4- ^) • (1 + £) . 



□ 

Corollary 1. The PTAS SKP is stable according to DIST, but not super- 
stable according to DIST. 

Proof. The first assertion directly follows from 1. To see that SKP is not 
super-stable according to DIST it is sufficient to consider the following in- 
put:^ 

^1 > ^2 ? •••? ? '^^2? •••5 , (>2 , C‘2m j 

where Wi = W2 = ... = Wm, Ui = U2 = ..Mm, lUi = lii + 1, 6 = Y^iLl 
Cl = C 2 = ... = Cm — (1 S)wi and Cm-\-l — Cm-\-2 ~ ••• — C2m (1 H" S)ui. 

□ 

We see that the PTAS SKP is stable according to DIST, but this does 
not suffice to get a PTAS for KFs for any (5 > 0. This is because in the 
approximation ratio we have the additive factor S-{2-\-S) that is independent 
on £. In what follows we change the PTAS SKP a little bit in such a way 
that we obtain a PTAS for every KP< 5 , (J > 0. 

PTAS MOD-SKP 

Input: Positive integers Cl, Cn for some n E IN and some 

positive real number £, 1 > £ > 0. 

Step 1: Sort For simplicity we may assume ^ for 

i = 1, ...,n — 1. 

Step 2: Set k — fl/£]. 

Step 3: The same as Step 3 of PTAS SKP, but the greedy procedure follows 
the ordering of the wi's of Step 1. 

Output: The best V constructed in Step 3. 

Let MOD-SKPe denote the algorithm given by PTAS MOD-SKP for a 
fixed £ > 0. 

^ Note, that m should be chosen to be essentially larger than for a given e. 
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Lemma 2. For every e, 1 > e > 0 and every S > 0, MOD-SKP^ is an 
e • (1 -h 6^ - approximation algorithm for SKs. 

Proof Let U = C {l,2,...,n}, where Wi^ < < ... < Wi^, be 

an optimal solution for the input I = wi, ...^Wn,b,ci, 

If I < k then MOD-SKP^ provides an optimal solution. 

If / > A:, then we consider a T' = {zi,Z 2 , jat+i, •••, jife+r} as a greedy 
extension of T = {«i,i 25 •••jU}* Again, we distinguish two possibilities: 



1. Let '^3 ^ Now, we show that this contradicts the 

optimality of U. Both, cost{U) and cost{T') contain Ci ^ . For the rest 
T' contains the best choice of Wi^s according to the cost of one weight 
unit. The choice of U per one weight unit cannot be better. So, cost(U) < 
cost{T'). 

2. Let d = ~ '^3 ^ Because of the optimal choice of T' 

according to the cost per one weight unit, the cost c of the first part of 
U with the weight YLj^T' ^^st cost{V). 

Since U and T‘ contain the same k indices ii,« 2 j •••jU and Wi ^ , are 

the largest weights in both U and T', the same same consideration as in 
the proof of Lemma 1 yields d < cost{U) < c + d-(l-h(5). 

Thus, 



cost{U) - cost{V) ^ c + d • (1 + J) - costiT') 
cost{U) “ cost{U) 



Wi 



cost{U) 



cost{U) 



= e-{l + 6)'^ 



□ 

We observe that the collection of MOD-SKPe algorithms is a PTAS for 
every KPg with a constant <5 > 0 (independent of the size 2n+ 1 of the input). 

Corollary 2. MOD-SKP is super-stable according to DIST. 

4 Conclusion and Discussion 

In the previous sections we have introduced the concept of stability of ap- 
proximations. Here we discuss the potential applicability and usefulness of 
this concept. 

Using this concept, one can establish positive results of the following 
types: 

1. An approximation algorithm or a PTAS can be successfully used for a 
larger set of inputs than the set usually considered (see Lemma 1). 
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2. We are not able to successfully apply a given approximation algorithm 
A (a PTAS) for additional inputs, but one can simply modify A to get a 
new approximation algorithm (a new PTAS) working for a larger set of 
inputs than the set of inputs of A (see Lemma 2). 

3. To learn that an approximation algorithm is unstable for a distance mea- 
sure could lead to the development of completely new approximation 
algorithms that would be stable according to the considered distance 
measure. 

The following types of negative results may be achieved: 

4. The fact that an approximation algorithm is unstable according to all 
’’reasonable” distance measures and so that its use is really restricted to 
the original input set. 

5. Let Q = {Uj, i7o, L, L/, M, cost^ goal) G NPO be well approximable. If, 

for a distance measure D and a constant r, one proves the nonexistence of 
any approximation algorithm for Qr,n = L, Roundr^D{Li)^ M, 

cost, goal), then this means that the problem Q is ’’unstable” according 
to D. 

Thus, using the notion of stability one can search for a spectrum of the 
hardness of a problem according to the set of inputs. For instance, consider- 
ing a hard problem like TSP one could get an infinite sequence of input lan- 
guages Lo, Li, L‘ 2 , ... given by some distance measure, where e^(n) is the best 
achievable relative error for the language L^. Results of this kind can essen- 
tially contribute to the study of the nature of hardness of specific computing 
problems, and to a sharp determination of the border between tractable and 
intractable optimization problems. 
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Some Examples of Average-case Analysis 
by the Incompressibility Method 

Tao Jiang, Ming Li, and Paul Vitanyi 



Summary. The incompressibility method is an elementary yet powerful proof tech- 
nique. It has been used successfully in many areas, including average-case analysis 
of algorithms [14]. In this expository paper, we include several new simple average- 
case analyses to further demonstrate the utility and elegance of the method. 



1 Introduction 



The incompressibility of individual random objects yields a simple but pow- 
erful proof technique, namely the incompressibility method. This method is a 
versatile tool that can be used to prove lower bounds on computational prob- 
lems, to obtain combinatorial properties of concrete objects, and to analyze 
the average-case complexity of algorithms. Since the early 1980’s, the in- 
compressibility method has been successfully used to solve many well-known 
questions that had been open for a long time and to supply new simplified 
proofs for known results. A comprehensive survey can be found in [14]. 

In this short expository paper, we use four simple examples of diverse 
topics to further demonstrate how easy the incompressibility method can be 
used to obtain (upper and lower) bounds which are useful in the domain 
of average-case analysis. The topics covered in this paper are well-known 
ones such as sorting, matrix multiplication, longest common subsequences, 
and majority finding. The proofs that we choose to include are not difficult 
ones and all the results are known before. However, our new proofs are much 
simpler than the old ones and are easy to understand. More such new proofs 
are contained in [5,13]. 

To make the paper self-contained, we give an overview of Kolmogorov 
complexity and the incompressibility method in the next section. We then 
consider the four diverse problems, namely sorting, boolean matrix multipli- 
cation, longest common subsequences, and majority finding, in four separate 
sections. 
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2 Kolmogorov Complexity and the Incompressibility 
Method 

We use the following notation. Let x be a finite binary string. Then l(x) 
denotes the length (number of bits) of x. In particular, /(e) = 0 where e 
denotes the empty word. 

We can map {0, 1}* one-to-one onto the natural numbers by associating 
each string with its index in the length-increasing lexicographical ordering 

(e, 0), (0, 1), (1, 2), (00, 3), (01, 4), (10, 5), (11, 6), . . . . (1) 

This way we have a binary representation for the natural numbers that is 
different from the standard binary representation. It is convenient not to 
distinguish between the first and second element of the same pair, and call 
them “string” or “number” arbitrarily. As an example, we have /(7) = 00. 
Let a:, y, G jV, where J\f denotes the natural numbers. Let Tq,Ti,... be a 
standard enumeration of all Turing machines. Let (•, •) be a standard one-one 
mapping from M x M to M, for technical reasons chosen so that l{{x,y)) = 
l{y) + 0{l{x)). 

Informally, the Kolmogorov complexity, [15], of x is the length of the short- 
est effective description of x. That is, the Kolmogorov complexity C{x) of a 
finite string x is simply the length of the shortest program, say in FORTRAN 
(or in Turing machine codes) encoded in binary, which prints x without any 
input. A similar definition holds conditionally, in the sense that C{x\y) is 
the length of the shortest binary program which computes x on input y. 
Kolmogorov complexity is absolute in the sense of being independent of the 
programming language, up to a fixed additional constant term which depends 
on the programming language but not on x. We now fix one canonical pro- 
gramming language once and for all as reference and thereby C(). For the 
theory and applications, as well as history, see [14]. A formal definition is as 
follows: 

Definition 1. Let U be an appropriate universal Turing machine such that 

U{{{i,p),y)) = Ti{{p,y)) 

for all i and {p,y). The conditional Kolmogorov complexity of x given y is 
C{x\y) = min {/(p) : U{(p,y)) = x}. 

The unconditional Kolmogorov complexity of x is defined as C{x) := C{x\e). 

By a simple counting argument one can show that whereas some strings 
can be enormously compressed, the majority of strings can hardly be com- 
pressed at all. For each n there are 2" binary strings of length n, but only 
S"=o 2‘ = 2" - 1 possible shorter descriptions. Therefore, there is at least 
one binary string x of length n such that C{x) > n. We call such strings 
incompressible. 
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Definition 2. For each constant c we say a string x is c-incompressihle if 
C{x) > l{x) — c. 

Strings that are incompressible (say, c-incompressible with small c) are 
patternless, since a pattern could be used to reduce the description length. 
Intuitively, we think of such patternless sequences as being random, and we 
use “random sequence” synonymously with “incompressible sequence.” It is 
possible to give a rigorous formalization of the intuitive notion of a random 
sequence as a sequence that passes all effective tests for randomness, see for 
example [14]. 

How many strings of length n are c-incompressible? By the same count- 
ing argument we find that the number of strings of length n that are c- 
incompressible is at least 2^ — 2”“^ H- 1. Hence there is at least one 0- 
incompressible string of length n, at least one-half of all strings of length 
n are 1-incompressible, at least three-fourths of all strings of length n are 
2-incompressible, ..., and at least the (1 — 1/2^) th part of all 2^ strings 
of length n are c-incompressible. This means that for each constant c > 1 
the majority of all strings of length n (with n > c) is c-incompressible. We 
generalize this to the following simple but extremely useful Incompressihility 
Lemma. 

Lemma 1. Let c be a positive integer. For each fixed y, every set A of cardi- 
nality m has at least m(l - 2”^) -f- 1 elements x with C{x\A, y) > [logmj — c. 

Proof. By simple counting. □ 

Definition 3. A prefix set, or prefix-free code, or prefix code, is a set of 
strings such that no member is a prefix of any other member. A prefix set 
which is the domain of a partial recursive function (set of halting programs for 
a Turing machine) is a special type of prefix code called a self- delimiting code 
because there is an effective procedure which reading left-to-right determines 
where a code word ends without reading past the last symbol. A one-to- 
one function with a range that is a self-delimiting code will also be called a 
self-delimiting code. 

A simple self-delimiting code we use throughout is obtained by reserving 
one symbol, say 0, as a stop sign and encoding a natural number x as 1^0. 
We can prefix an object with its length and iterate this idea to obtain ever 
shorter codes: 

Ei{x) = 

Thus, Ei(x) = and has length l{Ei{x)) = 2l{x)+l; E^ix) — Ei{l{x))x 

= and has length l(E 2 {x)) = l{x)+2l(l{x)) + l. We have for example 



f 1*0 for i = 0, 

1 Ei-i{l{x))x for i > 0. 



1 {E 3 {x)) < l{x) + log/(x) + 21oglog/(x) + 1. 
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Define the pairing function 

{x,y) = E2{x)y (3) 

with inverses (*)i, (-) 2 - This can be iterated to ((•,•),•). 

The Incompressibility Method. In a typical proof using the incompress- 
ibility method, one first chooses an individually random object from the class 
under discussion. This object is effectively incompressible. The argument in- 
variably says that if a desired property does not hold, then the object can be 
compressed. This yields the required contradiction. Then, since most objects 
are random, the desired property usually holds on the average. 

3 Lower Bound for Sorting 

We begin this paper with a very simple incompressibility proof for a well- 
known lower bound on comparison based sorting. 

Theorem 1. Any comparison based sorting algorithm requires logn! com- 
parisons to sort an array of n elements. 

Proof. Let A be any comparison based sorting algorithm. Consider permu- 
tation / of {1, . . . , n} such that 

C{I\A,P)>\ogn\ 

where P is a fixed program to be defined. Suppose A sorts / in m comparisons. 
We can describe I by recording the binary outcomes of the m comparisons, 
which requires a total of m bits. Let P be such a program converting m to I 
(given A). Thus, 

m>C(/|A,P) >logn! 

Hence, m > logn!. □ 

The above proof in fact also implies a lower bound of logn! - 2 logn on 
the average number of comparisons required for sorting. 

Corollary 1. Any comparison based sorting algorithm requires \ogn\—2\ogn 
comparisons to sort an array of n elements, on the average. 

Proof In the above proof, let I have Kolmogorov complexity: 

C{I\A,P) > logn! - logn 

Then we obtain that the (arbitrary) algorithm A requires 

m > logn! - logn 

comparisons on the permutation I. It follows from the Incompressibility 
Lemma that on the average, A requires at least 

jj^ 

(logn! - logn) 4- - • (n - 1) > logn! - 2 logn 

n n 

comparisons. □ 
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4 Average Time for Boolean Matrix Multiplication 

We begin with a simple (almost trivial) illustration of average-case analy- 
sis using the incompressibility method. Consider the well-known problem of 
multiplying two n x n boolean matrices A = (cnj) and B = (bij). Efficient 
algorithms for this problem have always been a very popular topic in the the- 
oretical computer science literature due to the wide range of applications of 
boolean matrix multiplication. The best worst-case time complexity obtained 
so far is due to Coppersmith and Winograd [7]. In 1973, O’Neil and 

O’Neil devised a simple algorithm described below which runs in 0{m?) time 
in the worst case but achieves an average time complexity of O(n^) [16]. 

Algorithm QuickMultiply(A, B) 

1. Let C = (cij) denote the result of multiplying A and B. 

2. For z := 1 to n do 

3. Let ji < • • • < jm be the indices such that =1, 1 < A: < m. 

4. For j := 1 to n do 

5. Search the list j, • ■ • , bj^j sequentially for a bit 1. 

6. Set Cij = 1 if a bit 1 is found, or cij = 0 otherwise. 

An analysis of the average-case time complexity of QuickMultiply is given 
in [16] using simple probabilitistic arguments. Here we give an analysis using 
the incompressibility method, to illustrate some basic ideas. 

Theorem 2. Suppose that the elements of A and B are drawn uniformly and 
independently. Algorithm QuickMultiply runs in 0{n^) time on the average. 

Proof. Let n be a sufficiently large integer. Observe that the average time 
of QuickMultiply is trivially bounded between 0{n^) and O(n^). By the 
Incompressibility Lemma, out of the pairs of n x n boolean matrices, 
at least (n — 1)2^^ /n of them are log n-incompressible. Hence, it suffices to 
consider log n-incompressible boolean matrices. 

Take a log n-incompressible binary string x of length 2n‘^, and form two 
n X n boolean matrices A and B straightforwardly so that the first half of 
X corresponds to the row-major listing of the elements of A and the second 
half of X corresponds to the row-major listing of the elements of B. We show 
that QuickMultiply spends 0{n^) time on A and B. 

Consider an arbitrary z, where 1 < z < n. It suffices to show that the n 
sequential searches done in Steps 4 - 6 of QuickMultiply take a total of 0{n) 
time. By the statistical results on various blocks in incompressible strings 
given in Section 2.6 of [14], we know that at least rz/2 - 0{y/n\ogn) of these 
searches find a 1 in the first step, at least n/4 — 0(\/n log n) searches find a 1 
in two steps, at least n/S — 0{y/n logn) searches find a 1 in three steps, and 
so on. Moreover, we claim that none of these searches take more than 4 logn 
steps. To see this, suppose that for some 1 < j < n, bj^j = • > > = = 

0. Then we can encode x by listing the following items in a self-delimiting 
manner: 




Average-case Analysis by the Incompressibility Method 255 



1. A description of the above discussion. 

2. The value of i. 

3. The value of j. 

4. All bits of X except the bits 

This encoding takes at most 

0(1) 4- 2 log n H- 2n^ - 4 log n + 0(log log n) < 2n^ - log n 

bits for sufficiently large n, which contradicts the assumption that x is logn- 
incompressible. 

Hence, the n searches take at most a total of 

log n 

(n/2^ - 0{y/n logn)) • k) + (logn) • 0(y^n logn) • (41ogn) 

k=l 
log n 

< ( ^ knf2’’ + 0(log^ n\/nlogn) 
k=l 

= 0{n) + 0(log^ n^/nlogn) 

= 0{n) 

steps. This completes the proof. □ 

5 Expected Length of a Longest Common Subsequence 

For two sequences (i.e. strings) s = si . . . and t = ti . . . we say that 
s is a subsequence of t if for some ii < . . . < i^, sj = ti-. A longest common 
subsequence (LCS) of sequences s and f is a longest possible sequence u that 
is a subsequence of both s and t. For simplicity, we will only consider binary 
sequences over the alphabet S = (0, 1}. 

Let n be an arbitrary positive integer and consider two random strings s 
and t that are drawn independently from the uniformly distributed space of 
all binary string of length n. We are interested in the expected length of an 
LCS of s and t. Tight bounds on the expected LCS length for two random 
sequences is a well-known open question in statistics [17,19]. After a series 
of papers, the best result to date is that the length is between 0.762n and 
0.838n [6,8-10] . The proofs are based on intricate probablistic and counting 
arguments. Here we give simple proofs of some nontrivial upper and lower 
bounds using the incompressibility method. 

Theorem 3. The expected length of an LCS of two random sequences of 
length n is at most 0.867n -t- o(n). 

Proof Let n be a sufficiently large integer. Observe that the expected length 
of an LCS of two random sequences of length n is trivially bounded between 
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n/2 and n. By the Incompressibility Lemma, out of the 2‘^'^ pairs of binary 
sequences of length n, at least (n — l)2^”/n of them are log n-incompressible. 
Hence, it suffices to consider log n-incompressible sequences. 

Take a log n-incompressible string x of length 2n, and let s and t be the 
first and second halves of x respectively. Suppose that string u is an LCS of s 
and t. In order to relate the Kolmogorov complexity of s and t to the length 
of n, we re-encode the strings s and t using the string u as follows. (The idea 
was first introduced in [12].) 

We first describe how to re-encode s. Let the LCS u = u\U 2 • • • n^, where 
m = l{u). We align the bits of u with the corresponding bits of s greedily 
from left to right, and rewrite s as follows: 

S = a\Uia2U2 * • • amUm^' • 

Here ai is the longest prefix of s containing no wi, 0:2 is the longest substring 
of s following the bit u\ containing no U 2 ^ and so on, and s' is the remaining 
part of s after the bit Um- Thus does not contain bit t/f, for i = 1, . . . ,m. 
In other words, each ai is a unary string consisting of the bit complementary 
to Ui. We re-encode s as string: 

s{u) = 1 • • • Is'. 

Clearly, given u we can uniquely decode the encoding s{u) to obtain s. 
Similarly, the string t can be rewritten as 

t = 0 iUiI3-2U2 ■ ■ ■ , 

where each /3i is a unary string consisting of the bit complementary to Ui, 
and we re-encode t as string: 

t{u) = 

Hence, the string x can be described by the following information in the 
self-delimiting form: 

1. A description of the above discussion. 

2. The LCS u. 

3. The new encodings s{u) and t{u) of s and t. 

Now we estimate the Kolmogorov complexity of the above description of 
X. Items 1 and 2 take m + 0(1) bits. Since s{u) contains at least m I’s, it is 
ectsy to see by simple counting and Stirling approximation (see e.g. [14]) that 

C(s{u)) < log^ (”) + 0(1) 

i=m ^ ' 



<logn + log(^j +0(1) 

< 2 log n 4- n log n - m log m — (n — m) log(n — m) + 0(1) 
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The second step in the above derivation follows from the trivial fact that 
m > n/2. Similarly, we have 

C{t{u)) < 2 log n + n log n - m log m - (n - m) log(n - m) -h 0(1) 

Hence, the above description requires a total size of 

O(logn) -f m + 2nlogn — 2mlogm — 2(n — m) log(n — m). 

Let p = n/m. Since C(x) >2n — logn, we have 

2n — log n < 0(log n) -\- m 2n log n — 2m log m — 2{n — m) log(n — m) 

= O(logn) +pn — 2nplogp — 2n(l — p) log(l — p) 

Dividing both sides of the inequality by n, we obtain 

2 < o(l) -h p - 2plogp - 2(1 - p) log(l - p) 

Solving the inequality numerically we get p < 0.867 — o(l). □ 

Next we prove that the expected length of an LCS of two random se- 
quences of length n is at least 0.66666n — 0{y/nlogn). To prove the lower 
bound, we will need the following greedy algorithm for computing common 
subsequences (not necessarily the longest ones). 

Algorithm Zero-Major(s = • • • Sn, t * tn) 

1. Let u := e he the empty string. 

2. Set i := 1 and j := 1; 

3. Repeat steps 4-6 until i > n or j > n: 

4. If Si = tj then begin append bit S{ to string u] i := i 1; j := j 1 

end 

5. Elseif = 0 then j := j -h 1 

6. Else i := i 1 

7. Return string u. 

Theorem 4. The expected length of an LCS of two random sequences of 
length n is at least 0.66666n — 0{y/n logn). 

Proof. Again, let n be a sufficiently large integer, and take a logn-incom- 
pressible string x of length 2n. Let s and t be the first and second halves 
of X respectively. It suffices to show that the above algorithm Zero-Major 
produces a common subsequence u of length at least 0.66666n - 0{y/nlogn) 
for strings s and t. 

The idea is to encode s and t (and thus x) using information from the 
computation of Zero-Major on strings s and t. We consider the comparisons 
made by Zero-Major in the order that they were made, and create a pair of 
strings y and 2 ; as follows. For each comparison (si,tj) of two complementary 
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bits, we simply append a 1 to 2 /. For each comparison {si^tj) of two identical 
bits, append a bit 0 to the string y. Furthermore, if this comparison of iden- 
tical bits is preceded by a comparison {sif,tjf) of two complementary bits, 
we then append a bit 0 to the string z if i' = i — 1 and a bit 1 if j' = j — 1. 
When one string (5 or t) is exhausted by the comparisons, we append the 
remaining part (call this w) of the other string to 2 . 

As an example of the encoding, consider strings s = 1001101 and t = 
0110100. Algorithm Zero-Major produces a common subsequence 0010. The 
following figure depicts the comparisons made by Zero-Major, where a 
indicates a mismatch and a “ 1 ” indicates a match. 

s = 10 01101 

comparisons * | | | ♦ | * 

t = 01101 0 0 

Following the above encoding scheme, we obtain y = 101100101 and z = 

01100. 

It is easy to see that the strings y and z uniquely encode s and t and, 
moreover, / ( 2 /) + ? ( 2 ;) = 2n. Since C{yz) > C{x)—2\ogn > 2n — Slog n — 0(1), 
and C{z) <l{z) 0(1), we have 

c{y) > Kv) - Slog n - 0(1) 



Similarly, we can obtain 



C{z) > l{z) — Slogn — 0(1) 



and 

C{w) > l{w) — Slogn — 0(1) 

where w is the string appended to > 2 ; at the end of the above encoding. 

Now let us estimate the length of the common subsequence u produced 
by Zero-Major on strings s and t. Let #zeroes(s) and H^zeroes{t) be the 
number of O’s contained in s and t respectively. Clearly, u contains 
min{#^eroe5(5),#zeroe5(t)} O’s. From [14] (page 159), since both s and t 
are logn-incompressible, we know 

n/2 - 0{^/n logn) < i^zeroes(s) < n/2 -f 0{^/n^o^) 

n/2 - 0{yjn logn) < H^zeroes{t) < n/2 -f 0{y/n logn) 

Hence, the string w has at most 0{y/n logn) O’s. Combining with the fact 
that C{w) > l{w) — Slogn — 0(1) and the above mentioned result in [14], we 
claim 

l{w) < 0{^n logn). 

Since l{z) — l{w) = l[u), we have a lower bound on l{u): 



l{u) > l{z) - 0(\/nlogn). 
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On the other hand, since every bit 0 in the string y corresponds to a 
unique bit in the common subsequence we have l{u) > H^zeroes{y). Since 
C(y) > l(y) - 21ogn - 0(1), 

l{u) > H^zeroes{y) > l{y)/2 — 0{\/nlogn). 



Hence, 

Sl{u) > l{y) 4- l{z) - 0{y/n logn) > 2n - 0{yjn logn). 

That is. 



l{u) > 2n/3 — 0(>/n logn) « 0.66666n — 0{\/n\ogn). 



□ 

Our above upper and lower bounds are not as tight as the ones in [6,8- 
10]. Recently, Baeza- Yates and Navarro improved our analysis and obtained 
a slightly better upper of 0.860 [4]. It will be interesting to know if stronger 
bounds can be obtained using the incompressibility method by more clever 
encoding schemes. 

6 Average Complexity of Finding the Majority 

Let a: = xi • • • Xn be a binary string. The majority bit (or simply, the ma- 
jority) of X is the bit (0 or 1) that appears more than [n/2j times in x. The 
majority problem is that, given a binary string x, determine the majority of 
X. When x has no majority, we must report so. 

The time complexity for finding the majority has been well studied in the 
literature (see, e.g. [1-3,11,18]). It is known that, in the worst case, n — i/(n) 
bit comparisons are necessary and sufficient [2,18], where i/(n) is the number 
of occurrences of bit 1 in the binary representation of number n. Recently, 
Alonso, Reingold and Schott [3] studied the average complexity of finding the 
majority assuming the uniform probability distribution model. Using quite 
sophisticated arguments based on decision trees, they sh owed t hat on the 
average finding the majority requires at most 2n/3 - ^8n/97r + O(logn) 
comparisons and at least 2n/3 - y8n/97r -h 0(1) comparisons. 

In this section, we consider the average complexity of finding the majority 
and prove an upper bound tight up to the first major term, using simple 
incompressibility arguments. 

The following standard tournament algorithm is needed. 

Algorithm Tournament (x = xi • • • Xn) 

1. If n = 1 then return x\ as the majority. 

2. Elseif n = 2 then 
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3. If xi = X 2 then return xi as the majority. 

4. Else return “no majority”. 

5. Elseif n = 3 then 

6. If xi = X ‘2 then return x\ as the majority. 

7. Else return xs as the majority. 

8. Let y = e. 

9. For i := 1 to [n/2\ do 

10. If X 2 i-i = X 2 i then append the bit X 2 i to y. 

11. If n is odd and [n/2j is even then append the bit Xn to y. 

12. Call Tournament (y). 

Theorem 5. On the average, algorithm Tournament requires at most2n/2t-]r 
0(v^ comparisons. 

Proof. Let n be a sufficiently large number. Again, since algorithm Tour- 
nament makes at most n comparisons on any string of length n, by the 
Incompressibility Lemma, it suffices to consider running time of Tournament 
on ^-incompressible strings, where S < logn. Consider an arbitrary S < logn 
and let X = xi • • • Xn be a fixed (^-incompressible binary string. For any inte- 
ger m < n, let cr(m) denote the maximum number of comparisons required 
by algorithm Tournament on any (5-incompressible string of length m. 

We know from [14] that among the L^/2J bit pairs (xi,X 2 ), . . . , 
(^ 2 Ln/ 2 j-ij ^ 2 [n/ 2 j) that are compared in step 10 of Tournament, there are 
at least n/4 — 0{y/n6) pairs consisting of complementary bits. Clearly, the 
new string y obtained at the end of step 11 should satisfy 

C{y)>l{y)-S^O{l) 

Hence, we have the following recurrence relation for cr{m): 
cr{m) < [m/2\ -f cr{m/4 + 0{VmS)) 

By straightforward expansion, we obtain that 
a{n) < [n/2\ -f- cr(n/4 -f- 0(\/^)) 

< n/2 4- a{nf4 -j- 0(\/nJ)) 

< n/2 -f (n/8 4- 0(v^)/2) 4- cr(n/16 4- 0(\/nJ)/4 4- 0{^/{n6)|4)) 

= n/2 -h (n/8 4- 0(V^)/2) 4- cr(n/16 -h (3/4) • 0(V^)) 

< ... 

< 2n/3 4- 0{\/n5) 

Using the Incompressibility Lemma, we can calculate the average com- 
plexity of algorithm Tournament as: 

log n 

Y. ^(2ri/3 + 0(v^)) + -n = 2n/3 + O(v^) 

_ 1 ^ n 



□ 
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